Having recently discovered that Asterisk supports regular expressions (with some limitations), I got to wondering why the FreePBX-based GUI’s don’t make better use of them. For example, in FreePBX, Trixbox, Elastix, etc., the Outbound Route pages all support entering patterns that are then directly used in Asterisk dialplans, in a most inefficient manner. Let me give you an example. Let’s say you have created an Outbound Route for toll-free calls, and in the “Dial Patterns that will use this Route” section you have entered these patterns:
1800NXXXXXX
1844NXXXXXX
1855NXXXXXX
1866NXXXXXX
1877NXXXXXX
1888NXXXXXX
800NXXXXXX
844NXXXXXX
855NXXXXXX
866NXXXXXX
877NXXXXXX
888NXXXXXX
Let’s further say that you have specified three trunks in the “Trunk Sequence for Matched Routes” section (this is actually conservative given that there are a number of ways you can route toll-free calls for free). Anyway, this is an actual representative context that would be generated in extensions_additional.conf:
[outrt-5] ; TollFreeinclude => outrt-5-custom
exten => _1800NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _1800NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _1800NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _1800NXXXXXX,n,Set(_NODEST=)
exten => _1800NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _1800NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _1800NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _1800NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _1800NXXXXXX,n,Macro(outisbusy,)
exten => _1844NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _1844NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _1844NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _1844NXXXXXX,n,Set(_NODEST=)
exten => _1844NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _1844NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _1844NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _1844NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _1844NXXXXXX,n,Macro(outisbusy,)
exten => _1855NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _1855NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _1855NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _1855NXXXXXX,n,Set(_NODEST=)
exten => _1855NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _1855NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _1855NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _1855NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _1855NXXXXXX,n,Macro(outisbusy,)
exten => _1866NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _1866NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _1866NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _1866NXXXXXX,n,Set(_NODEST=)
exten => _1866NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _1866NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _1866NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _1866NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _1866NXXXXXX,n,Macro(outisbusy,)
exten => _1877NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _1877NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _1877NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _1877NXXXXXX,n,Set(_NODEST=)
exten => _1877NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _1877NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _1877NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _1877NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _1877NXXXXXX,n,Macro(outisbusy,)
exten => _1888NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _1888NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _1888NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _1888NXXXXXX,n,Set(_NODEST=)
exten => _1888NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _1888NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _1888NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _1888NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _1888NXXXXXX,n,Macro(outisbusy,)
exten => _800NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _800NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _800NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _800NXXXXXX,n,Set(_NODEST=)
exten => _800NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _800NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _800NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _800NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _800NXXXXXX,n,Macro(outisbusy,)
exten => _844NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _844NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _844NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _844NXXXXXX,n,Set(_NODEST=)
exten => _844NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _844NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _844NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _844NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _844NXXXXXX,n,Macro(outisbusy,)
exten => _855NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _855NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _855NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _855NXXXXXX,n,Set(_NODEST=)
exten => _855NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _855NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _855NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _855NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _855NXXXXXX,n,Macro(outisbusy,)
exten => _866NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _866NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _866NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _866NXXXXXX,n,Set(_NODEST=)
exten => _866NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _866NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _866NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _866NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _866NXXXXXX,n,Macro(outisbusy,)
exten => _877NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _877NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _877NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _877NXXXXXX,n,Set(_NODEST=)
exten => _877NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _877NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _877NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _877NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _877NXXXXXX,n,Macro(outisbusy,)
exten => _888NXXXXXX,1,Macro(user-callerid,SKIPTTL,)
exten => _888NXXXXXX,n,Noop(Calling Out Route: TollFree)
exten => _888NXXXXXX,n,Set(MOHCLASS=${IF($[“${MOHCLASS}”=””]?default:${MOHCLASS})})
exten => _888NXXXXXX,n,Set(_NODEST=)
exten => _888NXXXXXX,n,Macro(record-enable,${AMPUSER},OUT,)
exten => _888NXXXXXX,n,Macro(dialout-trunk,7,${EXTEN},)
exten => _888NXXXXXX,n,Macro(dialout-trunk,10,${EXTEN},)
exten => _888NXXXXXX,n,Macro(dialout-trunk,8,${EXTEN},)
exten => _888NXXXXXX,n,Macro(outisbusy,)
; end of [outrt-5]
That is 112 lines of dialplan generated by only 12 lines of pattern in the Outbound Route and three trunk selections. If you have more patterns and/or more trunks, the generated dialplan can grow MUCH larger. At one point I had a dialplan so large that whenever I did an “orange bar” reload, it took two to three minutes to finish reloading. Clearly, there is something wrong here.
Now, let’s suppose that instead of, or better yet, in addition to the current Outbound Route pages, we had pages called “Outbound Route – Regular Expression”. It would eliminate the “Dial Patterns that will use this Route” section, replacing it with a “Regular Expression to select this Route” text box, and it would also have a text box that says “Extensions allowed to use this route (separated by the | character – RegEx patterns may be used).
Let’s say that the contents of the first text box are ^1?8(00|44|55|66|77|88)[2-9][0-9]{6}$ and are stored in variable string1. And let’s say that the contents of the second text box are 10[0-4]X|106[5-8] (specifying that extensions 1000-1049 and 1065-1068 may use this route) and are stored in string 2. Assuming that the part of the dialplan that processes this were in some kind of macro, and the called number is in the variable OUTNUM, you could perhaps do something like this:
exten => s,n,ExecIf($[${REGEX(“${string2}” ${CALLERID(num)})} != 1]?MacroExit())
exten => s,n,ExecIf($[${REGEX(“${string1}” ${OUTNUM})} != 1]?MacroExit())
exten => s,n,Macro(dialout-trunk,7,${OUTNUM},)
exten => s,n,Macro(dialout-trunk,10,${OUTNUM},)
exten => s,n,Macro(dialout-trunk,8,${OUTNUM},)
exten => s,n,Macro(outisbusy,)
The first line bails out of the macro if the Caller ID doesn’t match one of the permitted extensions (this would assume you by default use a regex that matches any extension on your system, for example ^[0-9]{4}$ would match any 4 digit extension number. If you don’t assume this, then you’d need an extra bit of code to skip a null value in string2).
The second line bails if the called number doesn’t match the pattern in string1.
If the called number does match on the regular expression for this route, and the call is from a permitted extension, then the three trunks are tried. Note that now you only need to list each of the three trunks once.
Granted this doesn’t account for every line in the original context, but the other lines could also be transferred into the macro, with a bit of thought. Wouldn’t this be simpler for those who know how to use regular expressions? And, it has the potential to generate MUCH shorter dialplans.
Those who understand the power of regular expressions and who have long or complicated dialplans would appreciate the ability to use regular expressions! If anyone implements this, don’t be stingy with the maximum string length in the Regular Expression text boxes; I guarantee you that 256 characters is not enough, and neither is 1,024 characters! At a MINIMUM I would say that you need to allow a 4,096 character (4K) regular expression, and while I personally can’t conceive of anyone exceeding that, my bet is that sooner or later someone will. Remember, you are potentially saving a whole bunch of memory and drive space by allowing regular expressions, so you don’t have to be miserly on the space allotted for the expressions themselves.
Related articles
- Did you know that Asterisk has the ability to evaluate Regular Expressions, though not in the same way as Perl or FreeSWITCH? (tech.iprock.com)
- How to use the F – PBX [macro-dialout-trunk-predial-hook] macro and regular expressions to blacklist or whitelist outgoing calls on all trunks (tech.iprock.com)
- How to send various types of notifications on an incoming call in FreePBX (tech.iprock.com)
- How to force FreePBX to immediately retry the same trunk again if it fails the first time even though the FreePBX developers apparently don’t want you to be able to do that (tech.iprock.com)
Regular Expressions are very powerful. But generally only advanced users use Regular Expressions. I assume that they are not supported in the GUI for a few reasons:
1. Not a huge demand
2. Even if there were a demand, there are other features in more demand
3. Could be a nightmare to support with users that don’t understand it
Robert, I honestly don’t believe the FreePBX developers give a flying f— about demand, unless perhaps the entity making the demand is backing it up with some cash. If they cared at all about demand, they would have brought back the free-form entry of outbound route and trunk patterns, at least as an option. Of course, they will now say that there is not any demand for that, but to whatever extent that’s true it’s only because they made it very clear they would not go back to the old method, so a lot of folks just gave up asking. “Ain’t no point in talking when there’s nobody listening”, as the Rod Stewart song said.
As for the “support nightmare”, they could always offer both types of Outbound Routes (two different menu selections) and if you go into the one for Regular Expressions, they could always put a notice at the top that reads something like this:
This module is for use by expert users only. Note that Asterisk regular expressions are not the same as those in certain other software you may be familiar with, such as Perl. We will not offer free assistance in constructing or debugging regular expressions. If you are not a regular expressions expert OR you do not understand the difference between what Asterisk allows in a regular expressions and what other software allows, you should use the other Outbound Routes module. Note that there is NO SYNTAX CHECKING on the regular expressions you enter here, and that a mal-formed expression could cause your PBX to stop functioning. DO NOT USE THIS MODULE IF YOU DON’T KNOW WHAT YOU ARE DOING AND ARE UNWILLING TO PAY FOR SUPPORT.
Something like that should be sufficiently frightening enough to scare off the new users that just want to experiment, but also makes it clear that if you break your system by using this module it’s going to cost you to get it fixed. Thus the “nightmare” is either eliminated, or turned into an avenue of revenue generation. Not that I really want to give the FreePBX people any ideas about soaking users for money, but in this case it might be justified because of your point #3, which IS very valid. Of course, I expect that people who know how to use the Regular Expressions would probably still post some sample regex’s in various forums, so it’s not as though the people who want to use it would necessarily have to pay anything.
Perhaps the module for this should only be offered as an add-on that you’d need to seek out and install, but my only concern about that is that the developers have this nasty habit of “breaking” add-on modules that people have come to rely upon in new releases (Swiss Army Knife being a prime example).
Anyway, I think once people learned to use the regular expressions and realized how much shorter and faster their dialplans could be, you’d see the demand rise. I never realized how much they simplify things until I started playing around with them a bit (particularly when I was developing the method shown in How to use the FreePBX [macro-dialout-trunk-predial-hook] macro and regular expressions to blacklist or whitelist outgoing calls on all trunks).
Just to play devil’s advocate, you have to be careful with regular expressions, especially if you use things like backreferences because the memory tradeoff can come in the form of a massive CPU performance hit instead, as the CPU has to calculate the effect of all the regular expressions in the dialplan to make sure the number doesn’t match any of them, when it could have done a quick lookup in a large memory table full of possibilities instead (assuming of course that Asterisk is doing something reasonably efficient in its data lookup structures in the dialplans, which it is in my experience. (Mine’s 32,000+ lines long, at the moment, and the system handles thousands of calls a day))
But if your patterns are relatively simplistic, I’m with you. 4K seems a bit much though.
Paul, point taken, and on a busy system that’s something that might cause concern. Part of the reason for wanting to do this, though, is to overcome the limitations of the GUI I still use (F—PBX a.k.a. “The Software I Have Come To Hate”) because at the moment my search for an alternative has been less than fruitful. While the developers of that software may be able to force some people to enter a large memory table full of lookup possibilities in the most inefficient manner possible, they really couldn’t do that with a regular expression. If you run raw Asterisk with no GUI you don’t have this issue, but unfortunately not all of us are able to do that (not and get all the features we might want, anyway).
Also, I’m pretty sure I’m not using backreferences because that’s a rather advanced concept that I don’t really understand all that well (read: basically not at all, but that’s partly because I would almost never have a need for them), plus, I have no idea if the regular expression processor in Asterisk would even handle them. It does not handle metacharacters like d as a substitute for “any digit”, so it would not surprise me if it doesn’t do backreferences.
I agree that 4K is a bit much, but my feeling was that if you make it 2K someone will almost certainly come up with a pattern that exceeds it. I already have one that’s just a bit over 1K, that contains all the USA area codes. I tend to err on the side of “make it large enough so that no one will bump up against the limit” because memory and hard drive storage space just aren’t that scarce anymore. Of course, if you are trying to make Asterisk run on something like a Raspberry Pi then it might be a bit more important to conserve bytes, but if one long line of regex will save you a thousand lines of dialplan or more then the tradeoff is still worth it, I would think.
I have recently started using Freeswitch while playing with PBX software. They make extensive use of RegEx in their configurations, most of which consist of XML hierarchies. It took me almost no time to configure a simple IVR using their system, and it works on both the Windows platform and Linux. I had to resort to Linux to implement the Text to Speech module, as it’s not configured for the windows installer yet, but there are several easy to follow installation guides for the linux version (I was not steeped in Linux from birth by any means…). I am currently trying to figure out how to use the Obi110 as a gateway from Freeswitch to the PSTN, which lead me to your blog. Hoping to glean info from your articles on using it as a gateway to Google Voice.
NOTICE: All comments above this one were imported from the original Michigan Telephone Blog and may or may not be relevant to the edited article above.