Welcome to our today’s guide related to Python regex. In this guide, we will discuss what are regular expressions in Python and how to use the “re” module.
Moreover, we will also explore advanced techniques, such as optimization techniques of the regular expression, lookahead assertions, and how regex can be utilized to improve data validation, streamline text parsing, and perform web scraping.
Source: How to Use Python Regular Expression with RE Library (GeeksVeda)
Grepping is awesome, as long as you don’t glob it up! This article covers some grep and regex basics.
Source: Globbing and Regex: So Similar, So Different | Linux Journal
Like Geeks recently published an interesting series on writing Bash shell scripts:
Today we are going to talk about bash script or shell scripting actually, they are called shell scripts in general but we are going to call them bash scripts because we are going to use bash among the other Linux shells. There are zsh, tcsh , ksh and other shells, you can review the basic Linux commands before starting on bash script programming.
Source: Bash Script Step By Step, You will love it
In the previous post, we talked about how to write a bash script. And we’ve seen how bash scripting is awesome. In this post, we continue to look at structured commands that control the flow of your shell scripts. You’ll see how you can perform repeating processes; this post demonstrates for loop, while in bash scripts
Source: Bash scripting the awesome guide Part2
Today we will know how to retrieve input from the user and deal with that input so our script becomes more interactive.
Source: Linux bash scripting the awesome guide part3
On the previous post we’ve talked about parameters and options in detail and today we will talk about something is very important in shell scripting which is input & output & redirection.
Source: Shell scripting the awesome guide part4
On the last post, we’ve talked about input and output and redirection in bash scripting. Now you start building some Linux bash scripts, you may wonder how to run and control them on your Linux system. The only way we’ve run scripts is directly from the command line interface in real-time mode. This isn’t the only way to run Linux bash scripts in Linux.
Source: Linux bash scripting the awesome guide part5
If you get tired writing the same blocks of code over and over in your bash script. It would be nice to just write the block of code once and refer to that block of code anywhere in your bash script without having to rewrite it.
The bash shell provides a feature allowing you to do just that called Functions.
Bash functions are blocks of script code that you assign a name to and reuse anywhere in your code. Anytime you need to use that block of code in your script, you simply use the function name you assigned it.
We are going to talk about how to create your own bash functions and how to use them in other shell scripts.
Source: Bash scripting the awesome guide part6 Bash functions
On the previous post we’ve talked about bash functions and how to use it from the command line and we’ve seen some other cool stuff I recommend you to review it, Today we will talk about a very useful tool for string manipulation called sed, sed Linux command is one of the most common tools that people use to work with text files like log files, configuration files, and other text files. If you perform any type of data manipulation in your bash scripts, you want to become familiar with the sed and gawk tools in this post we are going to focus on sed Linux command and see its ability to manipulate text which is very important step in our bash scripting journey
Source: 31+ Examples for sed Linux command in text manipulation
On the previous post we’ve talked about sed Linux command and we’ve seen many examples of using it in text processing and how it is good in this, nobody can deny that sed is very handy tool but it has some limitations, sometimes you need a more advanced tool for manipulating data, one that provides a more programming-like environment giving you more control to modify data in a file more robust. This is where awk command comes in.
The awk command or GNU awk specifically because there are many extensions for awk out there takes stream editing one step further than the sed editor by providing a programming language instead of just editor commands.
Source: 30 Examples for awk command in text processing
In order to successfully working with the Linux sed editor and the awk command in your shell scripts you has to understand regular expressions or in short regex and to be accurate in our case it is bash regex, since there are many engines for regex you can use and we here in this regex tutorial will use the shell regex and see the bash power in working with regex.
First, we need to understand what regex is then we will dive deep into using it
Source: Regex tutorial for Linux
In the last post, we’ve talked about regex and we see how to use them in sed and awk for text processing and we discussed before Linux sed command and awk command. During the series, we write small shell scripts but we didn’t mix things up, I think we should take a small step and write a shell script that can be some useful.
The main reason for learning to write a shell script is to be able to create your own Linux utilities. Understanding how to write useful and practical scripts is important.
However, sometimes it helps to do something fun to learn a concept or skill. The scripts in this post they can be lots of fun! And they help empower your script writing concepts.
Source: How to write practical shell script
In the previous post we’ve talked about writing practical shell scripts and we’ve seen how it is easy to write a shell script and we’ve used most of our knowledge we’ve discussed on the previous posts, today we are going to talk about a tool that does magic to our shell scripts,that tool is expect command or expect programming language. Expect command or expect programming language is a language that talks with your interactive programs or scripts that require user interaction for input. Expect works by expecting input, and upon receiving the expected input, the Expect script will send the response without any user interaction, just like magic.
Source: Expect command and how to automate shell scripts like magic
Having recently discovered that Asterisk supports regular expressions (with some limitations), I got to wondering why the FreePBX-based GUI’s don’t make better use of them. For example, in FreePBX, Trixbox, Elastix, etc., the Outbound Route pages all support entering patterns that are then directly used in Asterisk dialplans, in a most inefficient manner. Let me give you an example. Let’s say you have created an Outbound Route for toll-free calls, and in the “Dial Patterns that will use this Route” section you have entered these patterns:
1800NXXXXXX
1844NXXXXXX
1855NXXXXXX
1866NXXXXXX
1877NXXXXXX
1888NXXXXXX
800NXXXXXX
844NXXXXXX
855NXXXXXX
866NXXXXXX
877NXXXXXX
888NXXXXXX
Let’s further say that you have specified three trunks in the “Trunk Sequence for Matched Routes” section (this is actually conservative given that there are a number of ways you can route toll-free calls for free). Anyway, this is an actual representative context that would be generated in extensions_additional.conf:
[outrt-5] ; TollFree; end of [outrt-5]
That is 112 lines of dialplan generated by only 12 lines of pattern in the Outbound Route and three trunk selections. If you have more patterns and/or more trunks, the generated dialplan can grow MUCH larger. At one point I had a dialplan so large that whenever I did an “orange bar” reload, it took two to three minutes to finish reloading. Clearly, there is something wrong here.
Now, let’s suppose that instead of, or better yet, in addition to the current Outbound Route pages, we had pages called “Outbound Route – Regular Expression”. It would eliminate the “Dial Patterns that will use this Route” section, replacing it with a “Regular Expression to select this Route” text box, and it would also have a text box that says “Extensions allowed to use this route (separated by the | character – RegEx patterns may be used).
Let’s say that the contents of the first text box are ^1?8(00|44|55|66|77|88)[2-9][0-9]{6}$ and are stored in variable string1. And let’s say that the contents of the second text box are 10[0-4]X|106[5-8] (specifying that extensions 1000-1049 and 1065-1068 may use this route) and are stored in string 2. Assuming that the part of the dialplan that processes this were in some kind of macro, and the called number is in the variable OUTNUM, you could perhaps do something like this:
exten => s,n,ExecIf($[${REGEX(“${string2}” ${CALLERID(num)})} != 1]?MacroExit())
exten => s,n,ExecIf($[${REGEX(“${string1}” ${OUTNUM})} != 1]?MacroExit())
exten => s,n,Macro(dialout-trunk,7,${OUTNUM},)
exten => s,n,Macro(dialout-trunk,10,${OUTNUM},)
exten => s,n,Macro(dialout-trunk,8,${OUTNUM},)
exten => s,n,Macro(outisbusy,)
The first line bails out of the macro if the Caller ID doesn’t match one of the permitted extensions (this would assume you by default use a regex that matches any extension on your system, for example ^[0-9]{4}$ would match any 4 digit extension number. If you don’t assume this, then you’d need an extra bit of code to skip a null value in string2).
The second line bails if the called number doesn’t match the pattern in string1.
If the called number does match on the regular expression for this route, and the call is from a permitted extension, then the three trunks are tried. Note that now you only need to list each of the three trunks once.
Granted this doesn’t account for every line in the original context, but the other lines could also be transferred into the macro, with a bit of thought. Wouldn’t this be simpler for those who know how to use regular expressions? And, it has the potential to generate MUCH shorter dialplans.
Those who understand the power of regular expressions and who have long or complicated dialplans would appreciate the ability to use regular expressions! If anyone implements this, don’t be stingy with the maximum string length in the Regular Expression text boxes; I guarantee you that 256 characters is not enough, and neither is 1,024 characters! At a MINIMUM I would say that you need to allow a 4,096 character (4K) regular expression, and while I personally can’t conceive of anyone exceeding that, my bet is that sooner or later someone will. Remember, you are potentially saving a whole bunch of memory and drive space by allowing regular expressions, so you don’t have to be miserly on the space allotted for the expressions themselves.
DISCLAIMER: Just because I do something a certain way doesn’t mean YOU should, at least not without extensive testing to make sure that it will do what you want. I do not warrant that anything shown here will in any way save you from getting a high phone bill. If that should happen, consider it a happy coincidence, but if it doesn’t work as you expected, don’t come crying to me. I only offer what I think might work, it’s up to YOU to test it on YOUR system.
This is a followup to my article, Did you know that Asterisk has the ability to evaluate Regular Expressions, though not in the same way as Perl or FreeSWITCH? If you have not read that article, you may want to do that first. Both that article and this one are subject to future revision, if I discover more on this subject.
The problem I have always had on the two systems I help administer is that we don’t allow international calls that cost money. In the past I had resorted to putting a complete list of valid area code patterns in each user’s Outbound Route, and since I had to include patterns for both 11 digit and 10 digit calls, it grew to be a pretty long list. Fortunately, when adding a new user, it was easy to copy the list from an existing user’s outbound account, bring it into a text editor to edit the /extension suffix, and paste it into the new user’s Outbound Route. That worked in FreePBX 2.7, and in versions 2.8 and 2.9 if you use the Swiss Army Knife module, but that module won’t work with FreePBX 2.10 and above.
Unfortunately, as you are very well aware if you have read many of my previous posts, starting in FreePBX 2.8 the FreePBX developers changed the entry method for Outbound Route dial patterns, changing it from an incredibly easy process to enter a list of patterns into a painful, totally stupid method involving individual text boxes for each individual part of each line of each Outbound Route dial pattern. That one change was enough to make me hate FreePBX. The FreePBX developers have been totally unresponsive to those who don’t care for the new method (they pretty much try to pretend such users don’t even exist).
One thing I am NOT going to EVER do is enter a long list of dial patterns using their input method! Someone once said of me that I would spend weeks looking for a way around having to re-type something, when it might only take me a few hours to re-type it, and that’s true. I have even abandoned whole articles because my system crashed and there was no backup, and there was no damn way I was retyping what I had already typed. So in that spirit, I started looking for ways to get around having to enter all those dial patterns for each Outbound Route. I’ve used various stopgap measures including weird things like custom trunks, but that was an ugly solution and leaves extra entries in the Call Detail Records. When I found out that Asterisk could use Regular Expressions, I figured out this method, which is actually much faster, and gives me another way to bypass the limitations of FreePBX.
This uses the [macro-dialout-trunk-predial-hook] macro which is already present, but commented out, in /etc/asterisk/extensions_custom.conf. The thing you need to understand about this macro is that it is called AFTER most of the dial plan associated with both the outbound route and the trunk have been applied. So if you change the called number in either the Outbound Route Dial Patterns or the Trunk Dialed Number Manipulation Rules, those transformations will already have been applied by the time this macro is called. Also, if the calling number has been changed (by setting the Outbound Caller ID in the Trunk, for example), that change have been made by the time this macro is called. And finally, this macro is called when a call is about to go out on ANY trunk in your system (at least I think it does, although I haven’t tested it with every possible trunk type. I’m reasonably sure it is called whenever a call is destined for a SIP trunk, but I am not sure that it gets called if you use one of the other available types of trunks. It does NOT seem to get called when a call is destined for an ENUM trunk). If you don’t want it to act on calls from a certain trunk, you need to test for that and bail out of the macro (I show an example of that below).
The interesting thing about this is that because this macro is invoked so late in the progress of an outgoing call, you can use it as a last chance to look for “undesirable things” or to send notifications of certain happenings. For example, although I don’t show exactly how to do it below, you could have it send an instant message or an e-mail notification when someone calls a specific number, such as 911.
What I am showing here is what I have set up on one of my systems. This assumes that the outbound routes only contain the following Dial Patterns (not all Outbound Routes will contain all of these, and I assume that each user has their own outbound route, or that groups of users may share outbound routes):
1NXXNXXXXXX/ext
NXXNXXXXXX/ext
1aaa+NXXXXXX/ext
…where aaa is the user’s local area code for 7 digit dialing , and ext is the user’s extension (which could be a pattern to allow a group of extensions). In addition, if the user’s service allows the *67 code to be used to block outgoing Caller ID, then these patterns might also be used (with the same substitutions):
*671NXXNXXXXXX/ext
*67NXXNXXXXXX/ext
*671aaa+*67|NXXXXXX/ext
The way I have set it up also ignores any number under 10 digits (allowing it to go through) so if you have dial patterns that allow users to dial provider service codes, or extensions on another connected server, or some such thing, those calls won’t be affected — only calls of 10 digits or longer are checked (this is one reason I prepend the area code to 7 digit calls in the Outbound Routes). Note that there are long lines in the following so if you want to see them in their entirety, you need to cut and paste this into a text editor:
[macro-dialout-trunk-predial-hook]
; this macro intentially left blank so it may be safely overwritten for any custom
; requirements that an installation may have.
;
; MACRO RETURN CODE: ${PREDIAL_HOOK_RET}
; if set to "BYPASS" then this trunk will be skipped
;
exten => s,1,NoOp(Trunk ${OUT_${DIAL_TRUNK}} selected)
exten => s,n,ExecIf($[${LEN(${OUTNUM})}<10]?MacroExit())
exten => s,n,ExecIf($["${OUT_${DIAL_TRUNK}}"="SIP/TollFree-Future9"]?MacroExit())
exten => s,n,Set(regx=^([*]67)?1?900[0-9]{7}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?premium)
exten => s,n,Set(regx=^([*]67)?1?8(00|22|33|44|55|66|77|88)[2-9][0-9]{6}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?tollfree)
exten => s,n,Set(regx=^([*]67)?1?[2-9][0-9]{2}976[0-9]{4}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?premium)
exten => s,n,Set(regx=^([*]67)?1?(201|202|203|205|206|207|208|209|210|212|213|214|215|216|217|218|219|220|223|224|225|228|229|231|234|239|240|248|251|252|253|254|256|260|262|267|269|270|272|274|276|279|281|301|302|303|304|305|307|308|309|310|312|313|314|315|316|317|318|319|320|321|323|325|327|330|331|332|334|336|337|339|346|347|351|352|360|361|364|380|385|386|401|402|404|405|406|407|408|409|410|412|413|414|415|417|419|423|424|425|430|432|434|435|440|442|443|445|458|463|469|470|475|478|479|480|484|501|502|503|504|505|507|508|509|510|512|513|515|516|517|518|520|530|534|539|540|541|551|559|561|562|563|564|567|570|571|573|574|575|580|585|586|601|602|603|605|606|607|608|609|610|612|614|615|616|617|618|619|620|623|626|628|629|630|631|636|641|646|650|651|657|660|661|662|667|669|678|680|681|682|701|702|703|704|706|707|708|712|713|714|715|716|717|718|719|720|724|725|726|727|731|732|734|737|740|743|747|754|757|760|762|763|765|769|770|772|773|774|775|779|781|785|786|801|802|803|804|805|806|808|810|812|813|814|815|816|817|818|828|830|831|832|838|843|845|847|848|850|854|856|857|858|859|860|862|863|864|865|870|872|878|901|903|904|906|907|908|909|910|912|913|914|915|916|917|918|919|920|925|928|929|930|931|934|936|937|938|940|941|947|949|951|952|954|956|959|970|971|972|973|978|979|980|984|985|986|989)[2-9][0-9]{6}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?usa)
exten => s,n,Set(regx=^([*]67)?1?(204|226|236|249|250|289|306|343|365|367|403|416|418|431|437|438|450|506|514|519|548|579|581|587|604|613|639|647|705|709|778|780|782|807|819|825|867|873|879|902|905)[2-9][0-9]{6}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?canada)
exten => s,n,Set(regx=^0118835100[0-9]{8}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?inum)
exten => s,n,NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a NOT a permitted call)
exten => s,n,Jabbersend(asterisk,system_administrator_address@gmail.com,${CALLERID(num)} attempted to call ${OUTNUM} at ${STRFTIME(${EPOCH},,%l:%M:%S %p %Z on %A %B %e)})
exten => s,n,Playback(custom/sorry-outside-us-canada)
exten => s,n,Macro(hangupcall,)
exten => s,n,Set(PREDIAL_HOOK_RET=BYPASS)
exten => s,n,MacroExit()
exten => s,n(premium),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a premium number call - NOT perrmitted)
exten => s,n,Jabbersend(asterisk,system_administrator_address@gmail.com,${CALLERID(num)} attempted to call a premium number ${OUTNUM} at ${STRFTIME(${EPOCH},,%l:%M:%S %p %Z on %A %B %e)})
exten => s,n,Playback(silence/1&sorry-cant-let-you-do-that3)
exten => s,n,Macro(hangupcall,)
exten => s,n,Set(PREDIAL_HOOK_RET=BYPASS)
exten => s,n,MacroExit()
exten => s,n(usa),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a USA call)
exten => s,n,MacroExit()
exten => s,n(canada),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a Canada call)
exten => s,n,MacroExit()
exten => s,n(tollfree),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a toll free call)
exten => s,n,MacroExit()
exten => s,n(inum),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is an inum call)
exten => s,n,MacroExit()
exten => h,1,Macro(hangupcall,)
Now here’s a breakdown of the above, in which I try to explain what I have done. I will note at the outset that some may think I am using two lines where one will do, by putting the regular expression into a variable when I could just evaluate it directly in the REGEX function in the following line. However, when I was researching the use of regular expressions in Asterisk, I read on several pages that at least some versions of Asterisk have issues parsing a regular expression if it contains certain characters that also have a meaning to Asterisk, unless you put the regex into a string variable first and then use that in the REGEX function. I don’t know if that’s always true but I figured better safe than sorry, and besides, it’s much less confusing to me when I do it that way. But if you want to try combining the two-line pairs into one, feel free — it’s your system, and your headache if it doesn’t work. Also, remember that Asterisk doesn’t parse regular expressions in the same way that some other software (such as Perl) does, so if you try to simplify my regex patterns you might run into trouble. So with that said, let’s begin our tour.
This part of the macro is already present in extensions_custom.conf. You only need to uncomment the lines that are not obviously comments:
[macro-dialout-trunk-predial-hook]
; this macro intentially left blank so it may be safely overwritten for any custom
; requirements that an installation may have.
;
; MACRO RETURN CODE: ${PREDIAL_HOOK_RET}
; if set to "BYPASS" then this trunk will be skipped
;
exten => s,1,NoOp(Trunk ${OUT_${DIAL_TRUNK}} selected)
The next line is the first of many places we will use MacroExit() to bail out of the macro. This checks to see in the number called was 9 digits or less. We don’t want to block 911 calls, intra-company calls to other extensions, calls to another network that uses extensions of 9 digits or less, etc.:
exten => s,n,ExecIf($[${LEN(${OUTNUM})}<10]?MacroExit())
This bails out if the call is going out a specific trunk. In this case we don’t want to mess with toll-free calls going to Future9 because they are sent using a rather unusual pattern that would not be recognized by the regular expressions below.
exten => s,n,ExecIf($["${OUT_${DIAL_TRUNK}}"="SIP/TollFree-Future9"]?MacroExit())
Here’s an example of blocking a call that conforms to a specific pattern, in this case 1-900-XXX-XXXX or just 900-XXX-XXXX. It will block calls with or without the *67 (privacy) prefix, and with or without the leading “1”. If you don’t allow *67 calling then just remove the [*]67)? from the start of the expression. If it matches that pattern, it is transferred to the label “premium”, which we’ll get to in a bit:
exten => s,n,Set(regx=^([*]67)?1?900[0-9]{7}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?premium)
Here’s an example of allowing a call that matches a specific pattern, in this case a NANP toll-free call, again with or without the *67 (privacy) prefix (as is the case with most of these regular expressions — note that your outbound route(s) would also need to allow the *67 pattern). These calls are transferred to the label “tollfree”, which we’ll also talk about later:
exten => s,n,Set(regx=^([*]67)?1?8(00|22|33|44|55|66|77|88)[2-9][0-9]{6}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?tollfree)
Here we block calls that match the pattern 1-[2-9]XX-976-XXXX or just [2-9]XX-976-XXXX (with or without a *67 prepend) and send them to “premium”:
exten => s,n,Set(regx=^([*]67)?1?[2-9][0-9]{2}976[0-9]{4}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?premium)
Here we test to see if the call is going to any USA area code. This is a really long regex but it works. If the pattern matches, it transfers to the “usa” label
exten => s,n,Set(regx=^([*]67)?1?(201|202|203|205|206|207|208|209|210|212|213|214|215|216|217|218|219|220|223|224|225|228|229|231|234|239|240|248|251|252|253|254|256|260|262|267|269|270|272|274|276|279|281|301|302|303|304|305|307|308|309|310|312|313|314|315|316|317|318|319|320|321|323|325|327|330|331|332|334|336|337|339|346|347|351|352|360|361|364|380|385|386|401|402|404|405|406|407|408|409|410|412|413|414|415|417|419|423|424|425|430|432|434|435|440|442|443|445|458|463|469|470|475|478|479|480|484|501|502|503|504|505|507|508|509|510|512|513|515|516|517|518|520|530|534|539|540|541|551|559|561|562|563|564|567|570|571|573|574|575|580|585|586|601|602|603|605|606|607|608|609|610|612|614|615|616|617|618|619|620|623|626|628|629|630|631|636|641|646|650|651|657|660|661|662|667|669|678|680|681|682|701|702|703|704|706|707|708|712|713|714|715|716|717|718|719|720|724|725|726|727|731|732|734|737|740|743|747|754|757|760|762|763|765|769|770|772|773|774|775|779|781|785|786|801|802|803|804|805|806|808|810|812|813|814|815|816|817|818|828|830|831|832|838|843|845|847|848|850|854|856|857|858|859|860|862|863|864|865|870|872|878|901|903|904|906|907|908|909|910|912|913|914|915|916|917|918|919|920|925|928|929|930|931|934|936|937|938|940|941|947|949|951|952|954|956|959|970|971|972|973|978|979|980|984|985|986|989)[2-9][0-9]{6}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?usa)
Same as the previous except now we’re testing for Canadian area codes, and if successful transferring to the “canada” label:
exten => s,n,Set(regx=^([*]67)?1?(204|226|236|249|250|289|306|343|365|367|403|416|418|431|437|438|450|506|514|519|548|579|581|587|604|613|639|647|705|709|778|780|782|807|819|825|867|873|879|902|905)[2-9][0-9]{6}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?canada)
(NOTE: It should be noted that a few new area codes are added every year. You can see a list of the most recently added ones here: NPAs Introduced over the Last 10 Years. You should check this periodically and add the new USA and Canadian area codes to the appropriate sections above.)
Google Voice used to complete calls to iNum numbers for free, so this test allowed for those. Now I don’t use this section, but leave it here as an example. Back when these calls were free, this was the only “international” format call that was permitted on our system:
exten => s,n,Set(regx=^0118835100[0-9]{8}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${OUTNUM})} = 1]?inum)
If we get here, the call failed all the tests that would indicate it might be a permitted call. I play a custom recorded message that tells the caller that their call is not allowed, possibly because it’s not to a destination in the United States or Canada. Just prior to playing the message, I use the Jabbersend function to send myself an instant message showing that a call failed. I mainly do this so I can determine whether the caller really tried to dial a number that’s not permitted, or if there’s an error in my logic. For example, someone could be trying to place calls to a valid area code that somehow got omitted from the regular expression. Note that Jabbersend will only work if you have jabber.conf configured correctly AND the res_jabber.so module is loading when Asterisk starts up. If you have ever configured your server to place Google Voice calls using Asterisk’s jabber/gtalk support, those things are probably true, but it still may be a bit tricky to get working, however I’m not going to discuss that in this article — just search for Asterisk Jabbersend in a search engine if you need help:
exten => s,n,NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a NOT a permitted call)
exten => s,n,Jabbersend(asterisk,system_administrator_address@gmail.com,${CALLERID(num)} attempted to call ${OUTNUM} at ${STRFTIME(${EPOCH},,%l:%M:%S %p %Z on %A %B %e)})
exten => s,n,Playback(custom/sorry-outside-us-canada)
exten => s,n,Macro(hangupcall,)
exten => s,n,Set(PREDIAL_HOOK_RET=BYPASS)
exten => s,n,MacroExit()
One further note about the last three lines in the above section: Setting the PREDIAL_HOOK_RET variable and the MacroExit() are probably superfluous because the Macro(hangupcall,) will terminate the call, but I left them there for safety. Note that if you were to comment out the Macro(hangupcall,), the last two lines would execute, and then if more than one trunk was specified in the Outbound Route, it would keep retrying the call on each of those trunks (and the caller would hear the error message multiple times, and you’d get multiple IM’s). Although I can’t personally envision such a situation, there might be situations where trying the additional trunks would be desirable.
Next is where the call goes if the caller attempted to dial a premium (900 or 976) number. This is similar to the previous section, except the recording played to the caller is one second of silence followed by a standard system recording. The silence is necessary to keep the first word or two of the recording from being cut off on some clients. Also the IM I send to myself is slightly different. And the same comment applies on the last three lines. I know, I could have done a Goto to the same three lines in the previous section, but that would have only saved me two lines, and since the last two lines are superfluous anyway, if you are that tight on space you can just delete them.
exten => s,n(premium),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a premium number call - NOT perrmitted)
exten => s,n,Jabbersend(asterisk,system_administrator_address@gmail.com,${CALLERID(num)} attempted to call a premium number ${OUTNUM} at ${STRFTIME(${EPOCH},,%l:%M:%S %p %Z on %A %B %e)})
exten => s,n,Playback(silence/1&sorry-cant-let-you-do-that3)
exten => s,n,Macro(hangupcall,)
exten => s,n,Set(PREDIAL_HOOK_RET=BYPASS)
exten => s,n,MacroExit()
The next sections contain the usa, canada, tollfree, and inum labels. All they do is print an appropriate message to the CLI and exit. Rather than jump to these labels, I could have simply used statements of the form ExecIf($[${REGEX(“${regx}” ${OUTNUM})} = 1]?MacroExit()) and left out this entire next section, but I like the idea of being able to see which regex allowed the call in the CLI, and this also allows for the possibility of additional code. For example, I could insert code that keeps a running tally of calls broken down by their destination. I have no need to do anything like that, but maybe you do:
exten => s,n(usa),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a USA call)
exten => s,n,MacroExit()
exten => s,n(canada),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a Canada call)
exten => s,n,MacroExit()
exten => s,n(tollfree),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a toll free call)
exten => s,n,MacroExit()
exten => s,n(inum),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is an inum call)
exten => s,n,MacroExit()
This last line is important and was omitted from the default macro. It handles the case where the caller hangs up while this code is executing. Although that window is only a small fraction of second, bad things could happen if the caller does manage to hang up within that window and this statement isn’t present:
exten => h,1,Macro(hangupcall,)
Well, that’s how I’m doing it right now. It may not be how I do it two months from now, but this seems better than any method I have used thus far.
I will point out, for whatever it’s worth, that you COULD use an Asterisk dial command right in this macro. So, for example, let’s say you had a trunk that was real expensive at a certain time of the day but much cheaper during another. You COULD test to see if the call is being made within the “cheap” time and if so just exit, but if not you could execute a dial command that looks like the one normally executed, which is…
exten => s,n,Dial(${OUT_${DIAL_TRUNK}}/${OUTNUM},300,${DIAL_TRUNK_OPTIONS})
…except that in place of ${OUT_${DIAL_TRUNK}} you could specify a different trunk, for example SIP/OtherTrunk. And of course you could use this macro to change the dial trunk options for specific trunks, or to play a recording to callers if their call is going out over an “expensive” trunk. In fact, I can see where this macro could be used to get to get around some of the FreePBX syntax checking that has plagued me in the past.
It should probably be noted that if you watch these macros execute in the Asterisk CLI, every time one of them is executed you may see a WARNING message such as this:
[2012-11-22 11:48:12] WARNING[30588][C-00000055]: pbx.c:1585 pbx_exec: The application delimiter is now the comma, not the pipe. Did you forget to convert your dialplan? (Set(regx=^([\*]{2}1)?([\*]67)?W?1?8(00|22|33|44|55|66|77|88)[2-9][0-9]{6}$))This warning doesn’t apply in this situation because the pipe IS the correct character to use in a regular expression, when you are matching a single regular expression out of several possible regular expressions. See this page on Alternation with The Vertical Bar or Pipe Symbol for more information, with the caveat that Asterisk’s expression engine is a bit different from that used in other software, and therefore may not recognize syntax that would be perfectly valid in another application. Therefore, not all examples shown on various regular expression tutorials or reference pages will work. If you wish to try a particular syntax, be sure to do a test run or two and make sure it works. But, Asterisk’s regular expression engine DOES recognize the pipe character when used as shown in the above examples, but spits out these warning messages anyway, even though the pipe characters are part of the regular expression, and are not being used as application delimiters. You can safely ignore the warnings – they are a nuisance, but are meaningless in this particular situation. If you really don’t want to see those warnings, use a text editor to go to /etc/asterisk/asterisk.conf and uncomment this line:
;dontwarn = yes ; Disable some warnings.
This will disable some non-essential warnings such as this one. If it were me, I would have named this option “stopwhining” rather than “dontwarn”, because Asterisk does tend to complain a lot about things that don’t matter and that you can’t do anything about. 🙂
Once again, I will mention that articles in this blog (and especially one such as this) are subject to future editing as I discover more about the subject. If you find any neat uses for the [macro-dialout-trunk-predial-hook] macro (ESPECIALLY if they allow you to do something that FreePBX tries to stop you from doing, but that’s perfectly allowable in Asterisk), feel free to leave a comment!
I happened to be searching the Asterisk issue tracker a few minutes ago (trying to figure out why the Asterisk Blacklist suddenly stopped working for no apparent reason on one of my systems) and happened to notice this issue which had just been posted today:
REGEX function ignores shorthand character starting with backslash
Wait, you mean Asterisk can evaluate Regular Expressions? I didn’t think it had that capability. So, working off the example shown in that issue report (the working expression, naturally) I tried putting this in the FreePBX extensions_custom.conf [macro-dialout-trunk-predial-hook] context (I had to uncomment some existing lines and add others). This is just example code and not meant to do anything useful. NOTE: Many of the lines in code blocks below won’t fit on the screen due to the way WordPress formats this blog, but if you copy and paste the lines into a text editor you should see them in their entirety:
[macro-dialout-trunk-predial-hook]
; this macro intentially left blank so it may be safely overwritten for any custom
; requirements that an installation may have.
;
; MACRO RETURN CODE: ${PREDIAL_HOOK_RET}
; if set to "BYPASS" then this trunk will be skipped
;
exten => s,1,NoOp(Trunk ${OUT_${DIAL_TRUNK}} selected)
exten => s,n,Set(tftest=^1?8(00|22|33|44|55|66|77|88)[2-9][0-9]{6}$)
exten => s,n,GotoIf($[${REGEX("${tftest}" ${OUTNUM})} = 1]?tollfree)
exten => s,n,NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a NOT a toll free call)
exten => s,n,MacroExit()
exten => s,n(tollfree),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a toll free call)
exten => s,n,MacroExit()
If you place an outgoing call and watch the Asterisk CLI, you will see that if you place a call it knows what trunk was selected and what number the call came from, and prints that it is or is not a toll-free call. And I assume you could get other information about the call from other system variables. This could be handy for extended call routing, or for blocking of certain calls.
The thing to note here is that the dialplan only uses only one line to test for all the possible NANP toll-free area codes, both with and without a “1” prefix — you could do this without using a regular expression, but you’d need one line per area code to test. If you’ve ever looked at the FreePBX generated dialplan (in extensions_additional.conf) you know that much of the dialplan is lines upon lines of pattern specifications. I’ve mentioned in the past the problem of specifying every USA or Canadian area code in an outbound routes (so you can disallow calls to NANP points not covered by your calling plan) but it occurs to me that if FreePBX utilized regular expressions, it MIGHT be possible to condense many lines into one. Of course this would partly depend on whether there is a length limit on a regular expression, but in later testing I tried setting REGEX variables for the USA and Canada, like this:
This line that would create a regex that would test for USA calls (50 states + D.C.):
exten => s,n,Set(usatest=^1?(201|202|203|205|206|207|208|209|210|212|213|214|215|216|217|218|219|220|223|224|225|228|229|231|234|239|240|248|251|252|253|254|256|260|262|267|269|270|272|274|276|279|281|301|302|303|304|305|307|308|309|310|312|313|314|315|316|317|318|319|320|321|323|325|327|330|331|332|334|336|337|339|346|347|351|352|360|361|364|380|385|386|401|402|404|405|406|407|408|409|410|412|413|414|415|417|419|423|424|425|430|432|434|435|440|442|443|445|458|463|469|470|475|478|479|480|484|501|502|503|504|505|507|508|509|510|512|513|515|516|517|518|520|530|534|539|540|541|551|559|561|562|563|564|567|570|571|573|574|575|580|585|586|601|602|603|605|606|607|608|609|610|612|614|615|616|617|618|619|620|623|626|628|629|630|631|636|641|646|650|651|657|660|661|662|667|669|678|680|681|682|701|702|703|704|706|707|708|712|713|714|715|716|717|718|719|720|724|725|726|727|731|732|734|737|740|743|747|754|757|760|762|763|765|769|770|772|773|774|775|779|781|785|786|801|802|803|804|805|806|808|810|812|813|814|815|816|817|818|828|830|831|832|838|843|845|847|848|850|854|856|857|858|859|860|862|863|864|865|870|872|878|901|903|904|906|907|908|909|910|912|913|914|915|916|917|918|919|920|925|928|929|930|931|934|936|937|938|940|941|947|949|951|952|954|956|959|970|971|972|973|978|979|980|984|985|986|989)[2-9][0-9]{6}$)
Yes, that is all one single line (which you won’t see in its entirety if you don’t copy and paste the part you do see into a text editor, but it’s humongous)! But it seems to work if you use the ${usatest} variable instead of ${tftest} in the line containing the REGEX function! And here’s one for Canada that puts the expression into ${cdntest}:
exten => s,n,Set(cdntest=^1?(204|226|236|249|250|289|306|343|365|367|403|416|418|431|437|438|450|506|514|519|548|579|581|587|604|613|639|647|705|709|778|780|782|807|819|825|867|873|879|902|905)[2-9][0-9]{6}$)
Of course you could test for all three types of calls in the same block of code. Just remember that the code as shown above is handling calls destined for ALL your trunks, so if you only want it to operate on a single trunk you’ll have to test for that trunk only, and bail out if the call is headed for any other trunk.
For the moment all I will say is that this opens up interesting possibilities, including possibly new ways to get around some of the developer-imposed limitations of FreePBX, and also to streamline custom dialplan. Unfortunately it appears that Asterisk’s REGEX function support is not much used nor well documented, so you may have trouble finding other examples of working code that uses this function.
Also, Matt Jordan commented on the issue linked above, noting that “REGCOMP is built on the GNU extended regular expressions library (see regex.h). That library does not support the shorthand characters (see this comparison of regular expression libraries). As such, this is not a bug, but a limitation of the library that provides the regular expression functionality.” What this apparently means is that you cannot use something like d or [:digit:] to represent any digit, as you can do in other software that recognizes regular expressions, but [0-9] works just fine for that purpose. However, it would not be correct to say that backslashes can never be used, as I found these uses of REGEX in extensions_additional.conf:
exten => s-fixed,1,ExecIf($["${REGEX("^[+]?[0-9]+$" ${DB(RINGGROUP/${NODEST}/fixedcid)})}" = "1"]?Set(__TRUNKCIDOVERRIDE=${DB(RINGGROUP/${NODEST}/fixedcid)}))
exten => s-fixed,n,Return()
exten => s-extern,1,ExecIf($["${REGEX("^[+]?[0-9]+$" ${DB(RINGGROUP/${NODEST}/fixedcid)})}" == "1" & "${FROM_DID}" != ""]?Set(__TRUNKCIDOVERRIDE=${DB(RINGGROUP/${NODEST}/fixedcid)}))
exten => s-extern,n,Return()
exten => s-did,1,ExecIf($["${REGEX("^[+]?[0-9]+$" ${FROM_DID})}" = "1"]?Set(__REALCALLERIDNUM=${FROM_DID}))
exten => s-did,n,Return()
exten => s-forcedid,1,ExecIf($["${REGEX("^[+]?[0-9]+$" ${FROM_DID})}" = "1"]?Set(__TRUNKCIDOVERRIDE=${FROM_DID}))
exten => s-forcedid,n,Return()
Note the use of [+] in the REGEX expressions above. That “escapes” the + character, which otherwise would have a meaning in a regular expression, but for some reason the escaped character has to be enclosed in square brackets. This can lead to some non-obvious regex constructions. For example, look again at the line that creates a regex that would test for USA calls. It begins like this:
exten => s,n,Set(usatest=^1?(201...
The question mark after the “1” means it’s optional. But what if the provider also accepts calls prefixed with *67 to block the Caller ID? You could allow for that possibility like this in most software that accepts regular expressions, but not in Asterisk…
exten => s,n,Set(usatest=^(*67)?1?(201...
The backslash would escape the * character, and therefore the string *67 would be considered optional. But in Asterisk, you have to enclose the * in square brackets to make this work, like so:
exten => s,n,Set(usatest=^([*]67)?1?(201...
Anyway, I just wanted to bring this to your attention, in case that you (like me) were thinking that Asterisk doesn’t interpret regular expressions. It DOES, but not in the same manner as Perl, nor FreeSWITCH for that matter. If anyone knows of an online reference or “cheat sheet” for this particular variety of regular expression library, I’d appreciate the link.
Recent Comments