Welcome to our today’s guide related to Python regex. In this guide, we will discuss what are regular expressions in Python and how to use the “re” module.
Moreover, we will also explore advanced techniques, such as optimization techniques of the regular expression, lookahead assertions, and how regex can be utilized to improve data validation, streamline text parsing, and perform web scraping.
Source: How to Use Python Regular Expression with RE Library (GeeksVeda)
Grepping is awesome, as long as you don’t glob it up! This article covers some grep and regex basics.
Source: Globbing and Regex: So Similar, So Different | Linux Journal
Like Geeks recently published an interesting series on writing Bash shell scripts:
Today we are going to talk about bash script or shell scripting actually, they are called shell scripts in general but we are going to call them bash scripts because we are going to use bash among the other Linux shells. There are zsh, tcsh , ksh and other shells, you can review the basic Linux commands before starting on bash script programming.
Source: Bash Script Step By Step, You will love it
In the previous post, we talked about how to write a bash script. And we’ve seen how bash scripting is awesome. In this post, we continue to look at structured commands that control the flow of your shell scripts. You’ll see how you can perform repeating processes; this post demonstrates for loop, while in bash scripts
Source: Bash scripting the awesome guide Part2
Today we will know how to retrieve input from the user and deal with that input so our script becomes more interactive.
Source: Linux bash scripting the awesome guide part3
On the previous post we’ve talked about parameters and options in detail and today we will talk about something is very important in shell scripting which is input & output & redirection.
Source: Shell scripting the awesome guide part4
On the last post, we’ve talked about input and output and redirection in bash scripting. Now you start building some Linux bash scripts, you may wonder how to run and control them on your Linux system. The only way we’ve run scripts is directly from the command line interface in real-time mode. This isn’t the only way to run Linux bash scripts in Linux.
Source: Linux bash scripting the awesome guide part5
If you get tired writing the same blocks of code over and over in your bash script. It would be nice to just write the block of code once and refer to that block of code anywhere in your bash script without having to rewrite it.
The bash shell provides a feature allowing you to do just that called Functions.
Bash functions are blocks of script code that you assign a name to and reuse anywhere in your code. Anytime you need to use that block of code in your script, you simply use the function name you assigned it.
We are going to talk about how to create your own bash functions and how to use them in other shell scripts.
Source: Bash scripting the awesome guide part6 Bash functions
On the previous post we’ve talked about bash functions and how to use it from the command line and we’ve seen some other cool stuff I recommend you to review it, Today we will talk about a very useful tool for string manipulation called sed, sed Linux command is one of the most common tools that people use to work with text files like log files, configuration files, and other text files. If you perform any type of data manipulation in your bash scripts, you want to become familiar with the sed and gawk tools in this post we are going to focus on sed Linux command and see its ability to manipulate text which is very important step in our bash scripting journey
Source: 31+ Examples for sed Linux command in text manipulation
On the previous post we’ve talked about sed Linux command and we’ve seen many examples of using it in text processing and how it is good in this, nobody can deny that sed is very handy tool but it has some limitations, sometimes you need a more advanced tool for manipulating data, one that provides a more programming-like environment giving you more control to modify data in a file more robust. This is where awk command comes in.
The awk command or GNU awk specifically because there are many extensions for awk out there takes stream editing one step further than the sed editor by providing a programming language instead of just editor commands.
Source: 30 Examples for awk command in text processing
In order to successfully working with the Linux sed editor and the awk command in your shell scripts you has to understand regular expressions or in short regex and to be accurate in our case it is bash regex, since there are many engines for regex you can use and we here in this regex tutorial will use the shell regex and see the bash power in working with regex.
First, we need to understand what regex is then we will dive deep into using it
Source: Regex tutorial for Linux
In the last post, we’ve talked about regex and we see how to use them in sed and awk for text processing and we discussed before Linux sed command and awk command. During the series, we write small shell scripts but we didn’t mix things up, I think we should take a small step and write a shell script that can be some useful.
The main reason for learning to write a shell script is to be able to create your own Linux utilities. Understanding how to write useful and practical scripts is important.
However, sometimes it helps to do something fun to learn a concept or skill. The scripts in this post they can be lots of fun! And they help empower your script writing concepts.
Source: How to write practical shell script
In the previous post we’ve talked about writing practical shell scripts and we’ve seen how it is easy to write a shell script and we’ve used most of our knowledge we’ve discussed on the previous posts, today we are going to talk about a tool that does magic to our shell scripts,that tool is expect command or expect programming language. Expect command or expect programming language is a language that talks with your interactive programs or scripts that require user interaction for input. Expect works by expecting input, and upon receiving the expected input, the Expect script will send the response without any user interaction, just like magic.
Source: Expect command and how to automate shell scripts like magic
I happened to be searching the Asterisk issue tracker a few minutes ago (trying to figure out why the Asterisk Blacklist suddenly stopped working for no apparent reason on one of my systems) and happened to notice this issue which had just been posted today:
REGEX function ignores shorthand character starting with backslash
Wait, you mean Asterisk can evaluate Regular Expressions? I didn’t think it had that capability. So, working off the example shown in that issue report (the working expression, naturally) I tried putting this in the FreePBX extensions_custom.conf [macro-dialout-trunk-predial-hook] context (I had to uncomment some existing lines and add others). This is just example code and not meant to do anything useful. NOTE: Many of the lines in code blocks below won’t fit on the screen due to the way WordPress formats this blog, but if you copy and paste the lines into a text editor you should see them in their entirety:
[macro-dialout-trunk-predial-hook]
; this macro intentially left blank so it may be safely overwritten for any custom
; requirements that an installation may have.
;
; MACRO RETURN CODE: ${PREDIAL_HOOK_RET}
; if set to "BYPASS" then this trunk will be skipped
;
exten => s,1,NoOp(Trunk ${OUT_${DIAL_TRUNK}} selected)
exten => s,n,Set(tftest=^1?8(00|22|33|44|55|66|77|88)[2-9][0-9]{6}$)
exten => s,n,GotoIf($[${REGEX("${tftest}" ${OUTNUM})} = 1]?tollfree)
exten => s,n,NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a NOT a toll free call)
exten => s,n,MacroExit()
exten => s,n(tollfree),NoOp(${CALLERID(num)} is calling ${OUTNUM} which is a toll free call)
exten => s,n,MacroExit()
If you place an outgoing call and watch the Asterisk CLI, you will see that if you place a call it knows what trunk was selected and what number the call came from, and prints that it is or is not a toll-free call. And I assume you could get other information about the call from other system variables. This could be handy for extended call routing, or for blocking of certain calls.
The thing to note here is that the dialplan only uses only one line to test for all the possible NANP toll-free area codes, both with and without a “1” prefix — you could do this without using a regular expression, but you’d need one line per area code to test. If you’ve ever looked at the FreePBX generated dialplan (in extensions_additional.conf) you know that much of the dialplan is lines upon lines of pattern specifications. I’ve mentioned in the past the problem of specifying every USA or Canadian area code in an outbound routes (so you can disallow calls to NANP points not covered by your calling plan) but it occurs to me that if FreePBX utilized regular expressions, it MIGHT be possible to condense many lines into one. Of course this would partly depend on whether there is a length limit on a regular expression, but in later testing I tried setting REGEX variables for the USA and Canada, like this:
This line that would create a regex that would test for USA calls (50 states + D.C.):
exten => s,n,Set(usatest=^1?(201|202|203|205|206|207|208|209|210|212|213|214|215|216|217|218|219|220|223|224|225|228|229|231|234|239|240|248|251|252|253|254|256|260|262|267|269|270|272|274|276|279|281|301|302|303|304|305|307|308|309|310|312|313|314|315|316|317|318|319|320|321|323|325|327|330|331|332|334|336|337|339|346|347|351|352|360|361|364|380|385|386|401|402|404|405|406|407|408|409|410|412|413|414|415|417|419|423|424|425|430|432|434|435|440|442|443|445|458|463|469|470|475|478|479|480|484|501|502|503|504|505|507|508|509|510|512|513|515|516|517|518|520|530|534|539|540|541|551|559|561|562|563|564|567|570|571|573|574|575|580|585|586|601|602|603|605|606|607|608|609|610|612|614|615|616|617|618|619|620|623|626|628|629|630|631|636|641|646|650|651|657|660|661|662|667|669|678|680|681|682|701|702|703|704|706|707|708|712|713|714|715|716|717|718|719|720|724|725|726|727|731|732|734|737|740|743|747|754|757|760|762|763|765|769|770|772|773|774|775|779|781|785|786|801|802|803|804|805|806|808|810|812|813|814|815|816|817|818|828|830|831|832|838|843|845|847|848|850|854|856|857|858|859|860|862|863|864|865|870|872|878|901|903|904|906|907|908|909|910|912|913|914|915|916|917|918|919|920|925|928|929|930|931|934|936|937|938|940|941|947|949|951|952|954|956|959|970|971|972|973|978|979|980|984|985|986|989)[2-9][0-9]{6}$)
Yes, that is all one single line (which you won’t see in its entirety if you don’t copy and paste the part you do see into a text editor, but it’s humongous)! But it seems to work if you use the ${usatest} variable instead of ${tftest} in the line containing the REGEX function! And here’s one for Canada that puts the expression into ${cdntest}:
exten => s,n,Set(cdntest=^1?(204|226|236|249|250|289|306|343|365|367|403|416|418|431|437|438|450|506|514|519|548|579|581|587|604|613|639|647|705|709|778|780|782|807|819|825|867|873|879|902|905)[2-9][0-9]{6}$)
Of course you could test for all three types of calls in the same block of code. Just remember that the code as shown above is handling calls destined for ALL your trunks, so if you only want it to operate on a single trunk you’ll have to test for that trunk only, and bail out if the call is headed for any other trunk.
For the moment all I will say is that this opens up interesting possibilities, including possibly new ways to get around some of the developer-imposed limitations of FreePBX, and also to streamline custom dialplan. Unfortunately it appears that Asterisk’s REGEX function support is not much used nor well documented, so you may have trouble finding other examples of working code that uses this function.
Also, Matt Jordan commented on the issue linked above, noting that “REGCOMP is built on the GNU extended regular expressions library (see regex.h). That library does not support the shorthand characters (see this comparison of regular expression libraries). As such, this is not a bug, but a limitation of the library that provides the regular expression functionality.” What this apparently means is that you cannot use something like d or [:digit:] to represent any digit, as you can do in other software that recognizes regular expressions, but [0-9] works just fine for that purpose. However, it would not be correct to say that backslashes can never be used, as I found these uses of REGEX in extensions_additional.conf:
exten => s-fixed,1,ExecIf($["${REGEX("^[+]?[0-9]+$" ${DB(RINGGROUP/${NODEST}/fixedcid)})}" = "1"]?Set(__TRUNKCIDOVERRIDE=${DB(RINGGROUP/${NODEST}/fixedcid)}))
exten => s-fixed,n,Return()
exten => s-extern,1,ExecIf($["${REGEX("^[+]?[0-9]+$" ${DB(RINGGROUP/${NODEST}/fixedcid)})}" == "1" & "${FROM_DID}" != ""]?Set(__TRUNKCIDOVERRIDE=${DB(RINGGROUP/${NODEST}/fixedcid)}))
exten => s-extern,n,Return()
exten => s-did,1,ExecIf($["${REGEX("^[+]?[0-9]+$" ${FROM_DID})}" = "1"]?Set(__REALCALLERIDNUM=${FROM_DID}))
exten => s-did,n,Return()
exten => s-forcedid,1,ExecIf($["${REGEX("^[+]?[0-9]+$" ${FROM_DID})}" = "1"]?Set(__TRUNKCIDOVERRIDE=${FROM_DID}))
exten => s-forcedid,n,Return()
Note the use of [+] in the REGEX expressions above. That “escapes” the + character, which otherwise would have a meaning in a regular expression, but for some reason the escaped character has to be enclosed in square brackets. This can lead to some non-obvious regex constructions. For example, look again at the line that creates a regex that would test for USA calls. It begins like this:
exten => s,n,Set(usatest=^1?(201...
The question mark after the “1” means it’s optional. But what if the provider also accepts calls prefixed with *67 to block the Caller ID? You could allow for that possibility like this in most software that accepts regular expressions, but not in Asterisk…
exten => s,n,Set(usatest=^(*67)?1?(201...
The backslash would escape the * character, and therefore the string *67 would be considered optional. But in Asterisk, you have to enclose the * in square brackets to make this work, like so:
exten => s,n,Set(usatest=^([*]67)?1?(201...
Anyway, I just wanted to bring this to your attention, in case that you (like me) were thinking that Asterisk doesn’t interpret regular expressions. It DOES, but not in the same manner as Perl, nor FreeSWITCH for that matter. If anyone knows of an online reference or “cheat sheet” for this particular variety of regular expression library, I’d appreciate the link.
GiottoPress by Enrique Chavez
Recent Comments