Most guides to bash history shortcuts exhaustively list all of the shortcuts available to you.
The problem I always had with that was that I would use them once, and then glaze over as I tried out all the possibilities. Then I’d move onto my working day and completely forget them, retaining only the well-known !! trick I learned when I first started using bash.
So most never got committed to memory.
Here I outline the shortcuts I actually use every day.
If you’ve ever used a spreadsheet, you’ve probably used or seen functions for doing date math—in other words, taking one date and adding some number of days or months to it to get a new date, or taking two dates and finding the number days between them. The same thing can be done from the command line using the lowly date command, possibly with a little help from Bash’s arithmetic.
I call this the unofficial bash strict mode. This causes bash to behave in a way that makes many classes of subtle bugs impossible. You’ll spend much less time debugging, and also avoid having unexpected complications in production.
Ever wondered why programming in Bash is so difficult? Bash employs the same constructs as traditional programming languages; however, under the hood, the logic is rather different.
Every so often something really useful appears on Reddit, and this is such a case. You may encounter a situation where you want to execute the contents of a bash script, but not more frequently than every few seconds. A Reddit user wanted to know How to check if a command in .bashrc has been executed within last 10 seconds if yes don’t execute the command again. The response by Reddit user mdaffin is brilliant in its simplicity, and can be used in any bash script where you don’t want the contents executed too often:
Write a time stamp to some file, check said file before you run the command if now – timestamp > 10s run the command and update the timestamp.
EDIT: Like this (with modification times instead):
TS_FILE="$HOME/.cache/your-app-lock"
if [[ ! -f "$TS_FILE" ]] || [[ "$(expr "$(date +%s)" - "$(stat -c %Y "$TS_FILE")")" -gt 10 ]]; then
touch "$TS_FILE"
echo "running"
fi
You’d replace the echo "running" line with the part of the bash script you want to run only if it’s been 10 seconds since the last time the script was run, or whatever number of seconds you specify after the -gt. If the bash script actually outputs a file as part of its normal operation then you could specify that file in the TS_FILE= line; there would be no need to create a separate timestamp file (unless some other process could also modify that same file).
This doesn’t actually stop the bash script from running; it just prevents it from executing the part of the script that you don’t want executed too frequently. This could be very useful in a situation where without such protection, the too-frequent execution of the script might cause something undesirable to happen (such as getting locked out of an online site for hammering it with requests). Depending on the situation there may be other, perhaps even better ways to avoid this possibility, but in other cases this may indeed be the best approach.
This is a revision of the post, A Perl script to rewrite the "static" IP address in the FreePBX Asterisk SIP Settings when it is changed by your ISP, but modified to use a Bash script. Much of the explanatory text is directly copied, or in some cases heavily modified, from the earlier article, which in turn was taken (with permission) from the old Michigan Telephone blog after it went defunct. THIS SCRIPT IS STILL CONSIDERED EXPERIMENTAL – USE AT YOUR OWN RISK!!!
This post is going to be a bit long because I first need to explain the “why” behind this script, then the script itself and how to test it after installation. Please note that the screenshots below are from earlier versions of FreePBX, and you probably will be using a more modern GUI interface, but the settings shown should still appear.
If you are using a recent version of Asterisk and FreePBX you may be using the Asterisk SIP Settings module (under the “Settings” tab) to automatically set various SIP parameters. This module is a great help to those who don’t know what they are doing, but there is a trap for the unwary (and in this case it’s NOT the fault of FreePBX – it’s a longstanding bug in Asterisk that’s the problem).
At the top of the Asterisk SIP Settings configuration page, in the NAT Settings section, there are two options that can be set. The first is NAT and there are four possible choices:
yes = Always ignore info and assume NAT
no = Use NAT mode only according to RFC3581
never = Never attempt NAT mode or RFC3581
route = Assume NAT, don’t send rport
In theory, if you have a fixed IP address AND your Asterisk server is not behind an external router that does NAT translation, you should use “no” (and most of the rest of this article will not be relevant to you). This article is intended more for home and SOHO users that both have their Asterisk server behind a hardware router of some kind, and that get their broadband service from a company that occasionally changes their IP address without warning. For such users, the preferred setting is “yes”. I’m not enough of a networking guru to tell you under what circumstances one of the other settings might be appropriate (if you understand this stuff, feel free to leave a comment and enlighten us).
It’s the next set of settings that can get us into trouble. This is the IP Configuration and there are three possible choices:
Public IP
Static IP
Dynamic IP
If your IP address never changes AND you aren’t behind a hardware firewall then you can usually just set this to “Public IP” and let it go at that. You will not be asked to fill in any other values. But most users that are not in that situation will pick one of the other two choices, and this is where the problem arises. Conventional wisdom has it that if your ISP ever changes your IP address without advance warning (which is the case for most cable broadband and DSL users), you should use the Dynamic IP setting. In this case there is an auto-configure button that will fill out the fields for you, although you may need to fill in the Dynamic Host field yourself. This is the “External FQDN as seen on the WAN side of the router and updated dynamically, e.g. mydomain.dyndns.com” (as explained if you mouse over the words “Dynamic Host”). You can use a DynDNS address (or an address from a similar service) or an address you have purchased. But the problem is that for some users, THIS METHOD SIMPLY DOES NOT WORK.
If you try to use Dynamic IP and it won’t work for you, what happens is you will get all sorts of weird errors. You may get one way audio, some calls may disconnect for no apparent reason after about five seconds, and you will see other weird errors in your CLI. If you change this setting to “Static IP” and click the auto-configure button and then submit the changes, the problems magically go away – UNTIL your ISP changes your IP address, at which point you suddenly have no connectivity to the outside world. If you ask for help, everybody and their brother will tell you to use the Dynamic IP setting, and the minute you try that you’ll get all the weird errors again.
FreePBX: Asterisk SIP Settings page, Chan SIP Settings tab, NAT Settings (Static IP Option)FreePBX: Asterisk SIP Settings page, General SIP Settings tab, NAT Settings, showing static external IP address
So if that’s your situation, you need set a Static IP address as shown in the above two screenshots (this used to be all on one page, but now it’s under two separate tabs in the new interface), and you need this Bash script. Coupled with a cron job, it goes out and checks your IP address every five minutes and if it notices it has changed, it changes it in the MySQL database (same as if you entered it into the External IP text box on the Asterisk SIP Settings configuration page) and then reloads Asterisk. Therefore, you can use the Static IP method and it hopefully it will work reliably. If and when your IP address changes, you should only be down for about five to ten minutes at most (hopefully your broadband provider usually does such changes in the middle of the night!).
A word to the wise, do NOT enter anything into the “Override External IP” field under the “Chan SIP Settings” tab unless you are certain that you know what you are doing, or you may have “no audio” issues. You should only put your external IP address in the “External Address” field under the “General SIP Settings” tab.
Prerequisites:
You will need to use a Dynamic DNS service to keep track of your IP address if you want external extensions to be able to find your server on the Internet. It’s not required for this script to work, though, so I won’t say any more about that except to note that if you use a recent vintage hardware router, it probably has DDNS support built in.
The Script:
Note that WordPress MAY change apostrophes and quotes into “prettified” versions, and if it does that will totally mess up Bash. I’m going to put this in a preformatted text block so hopefully WordPress won’t change anything (it doesn’t appear that it has), but you never know. Also, don’t confuse backticks (`) with apostrophes (‘) – backticks are used around the word `key` in the script below.
This script was written for use with FreePBX 14; there are some minor changes that need to be made for earlier versions (for example, you may need to use kvstore in place of kvstore_Sipsettings where it appears in the script). If you are running an earlier version you can either use the Perl version of this script from A Perl script to rewrite the "static" IP address in the FreePBX Asterisk SIP Settings when it is changed by your ISP, or at least see that article to see an explanation of the changes.
There are long lines in this script that overflow the allotted display area, so you will probably want to copy and paste the entire script (except for the final three commented out lines if you don’t need it to update a Dynamic DNS; see below) into a text editor such as nano. Make sure you do copy the last line, which contains only “fi”, or it won’t work.
#!/bin/bash
# This program gets the current IP address (as assigned by the ISP) from
# OpenDNS and modifies the FreePBX Asterisk SIP settings if the external IP
# address has changed. Invoke it as cron job that runs every 5 minutes.
# THIS SCRIPT IS STILL CONSIDERED EXPERIMENTAL - USE AT YOUR OWN RISK!!!
user="user"
pass="pw"
check=$(dig +short myip.opendns.com @resolver1.opendns.com) || { echo "Problem getting current IP address"; exit 1; }
if [[ ! $check =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "Invalid IP address"; exit 1;
fi
readip=$(mysql asterisk -u $user -p$pass -se 'SELECT val FROM kvstore_Sipsettings where `key` = "externip"') || { echo "Can't read externip from MySQL asterisk database"; exit 1; }
if [ "$check" != "$readip" ]; then
# IP address has changed
# Send email
# echo "This is an automated message - please do not reply. It appears that our external IP address has been changed to $check" | mail -s "External IP address may have been changed" you@someaddress.com
# Save new IP address to Asterisk SIP settings
mysql_response=$(mysql asterisk -u $user -p$pass -se 'update kvstore_Sipsettings set val='\"$check\"' where `key`="externip" ')
# Reload Asterisk
/var/lib/asterisk/bin/module_admin reload
# Update freedns
# date > /tmp/freedns_uodate.log
# /bin/sleep 21 ; /usr/bin/wget --no-check-certificate -O - https://freedns.afraid.org/dynamic/update.php?uniqueupdatestring== >> /tmp/freedns_update.log 2>&1 &
fi
NOTES on the above script, including THINGS YOU MUST CHANGE:
Note the two bolded variables user and pw. These must be changed to the correct values for YOUR system. You will usually find these in one of two places. You can look in /etc/amportal.conf and look for the variables AMPDBUSER and AMPDBPASS — these will usually be near the bottom of the file in newer installs, in a “— CATEGORY: Bootstrapped or Legacy Settings —” section, but they can be anywhere in the file.
Another place they may be found is in the file /etc/freepbx.conf — in that file, look for lines similar to:
Those will give you the values to insert into the user and pw variables in the script. YOU MUST INSERT THE CORRECT VALUES OR THE SCRIPT WILL NOT WORK! By the way, if you have both of the above-mentioned files, make sure that the AMPDBUSER and AMPDBPASS variables are set to the same respective values in both files, otherwise your CDR Reports page may not work.
If you want an e-mail notification when your IP address has changed (which is recommended), uncomment the next line under “# Send email” and modify the email address appropriately (make sure you use one or more valid e-mail addresses; you use a comma to separate addresses if you use more than one). Note this will only work if your system is already configured to send outgoing email.
You can also use this script to update a Dynamic DNS; if you use freedns.afraid.org as your Dynamic DNS then you could uncomment the two lines under “# Update freedns” and change the update URL to the unique one for your account. Don’t uncomment those lines unless and until you have the correct update URL for YOUR freedns account!. If you don’t want this script to update a Dymanic DNS account (as might be the case if you have your router set to perform this function) then you can just omit the final three commented out lines of the script – make sure you DO include the “fi” at the end of the script, though!
Note that in two places in the script (the lines that access MySQL) the word `key` appears within backtick quotes. If you leave out the backticks, or change them to something else such as apostrophes, IT WILL NOT WORK. Backticks and apostrophes are NOT the same character!
Save your script to either the /root directory or the /var/lib/asterisk/agi-bin directory, or to another location of your choosing. I named it checkip.sh. You must make the script executable, for example:
chmod u+rx /var/lib/asterisk/agi-bin/checkip.sh
Of course you will specify the correct filename and directory. Now it’s time to test the script. From the Linux command prompt, navigate to the directory where you stored the script:
cd /var/lib/asterisk/agi-bin
Now run the script from the command prompt:
./checkip.sh
Hopefully you won’t see any error messages. Remember it’s going out to do a query to get your external IP address, so don’t get concerned if it takes a second or two. If you had an incorrect address stored in your FreePBX Asterisk SIP Settings configuration, it will take longer because it will reload the FreePBX configuration. The script has a check to make sure it only stores a real IP address (and not something invalid like an error message) in the database, so if it appears to not be working, make sure the underlying call to the dig command is returning a valid IP address. By the way, that IP address check is rather rudimentary, and really only checks that the address only contains four sets of numbers separated by three dots, so if the dig command were to somehow return a totally bogus, but valid looking IP address, it might not get caught, however that’s highly unlikely to happen. That said, if you know of a better way to validate an IP address (or for that matter, see a better way to do anything in this script), please leave a comment!
If you do get an error, check that you copied the entire script correctly (including the ending “fi” statement), and that you made all the necessary changes as indicated above. Also check to make sure that you made the script executable.
Setting up a cron job
Once it runs without errors, you will want to create a cron job so it runs automatically every five minutes. Do NOT run it more often than that, or the lookup service may ban your IP address, and you don’t want that to happen, and besides, it’s not polite to hog the resources of someone else’s server! And if you are running it on multiple servers at the same IP address, then adjust the polling speed so that the total polling from all servers doesn’t exceed once every five minutes. An occasional additional test is probably not an issue, but if you try to poll every minute you just might get banned!
The usual way to add a cron job is to run this command:
crontab -e
(If you’re not currently running as root use sudo crontab -e instead)
This will open a text editor showing your current cron jobs. Just add a new line to the bottom of the file with your new cron job. To run the script every five minutes, you could use something like this:
*/5 * * * * /var/lib/asterisk/agi-bin/checkip.sh
Or to be more specific as to when the script runs (this will run it exactly on the hour, at five minutes after the hour, at ten minutes after the hour, and so on):
Just save the changed file when you are finished. The alternate method is to use Webmin’s System | Scheduled Cron Jobs module to set up your cron job.
Final testing:
The easiest way to test to make sure this is all working is to wait until a time that there are no active calls on the system, then go to the Asterisk SIP Settings configuration page and change the External IP address to something invalid (just change the last digit of the current address and Submit Changes, then do the usual configuration reload). On the next five minute interval, the script should detect that the external IP address doesn’t match the one stored in the database, and it will write the correct value to the database and reload the FreePBX configuration. If you watch the Asterisk CLI during this time, you should actually see the reload take place. After that, if you go back to the Asterisk SIP Settings configuration page, the correct IP address should be there. To be extra safe, you should also view the contents of the file /etc/asterisk/sip_general_additional.conf and make sure that the externip= line shows the correct IP address.
Now you don’t have to worry about frantic calls from users at inopportune times because your ISP changed your IP address and none of the phones are working, and you also won’t have any of the problems associated with the Dynamic IP method!
NOTE: Again, this script should be considered experimental, and as usual, there are no warranties — we’re experimenters here, and sometimes we don’t catch all the bugs, especially on the first go around! However, I would assume that anyone who is running a “professional” installation would pay their ISP for a true static IP address (one that never changes), and therefore wouldn’t need this type of script.
You can write a bash script such that it receives arguments specified when the script is called from the command line. This method is used when a script has to perform a slightly different function depending on the values of input parameters (the arguments).
For example, you may have a script called “stats.sh” that performs a particular operation on a file, such as counting its words. If you want to be able to use that script on many files, it is best to pass the file name as an argument, so that you can use the same script for all the files to be processed.
Like Geeks recently published an interesting series on writing Bash shell scripts:
Today we are going to talk about bash script or shell scripting actually, they are called shell scripts in general but we are going to call them bash scripts because we are going to use bash among the other Linux shells. There are zsh, tcsh , ksh and other shells, you can review the basic Linux commands before starting on bash script programming.
In the previous post, we talked about how to write a bash script. And we’ve seen how bash scripting is awesome. In this post, we continue to look at structured commands that control the flow of your shell scripts. You’ll see how you can perform repeating processes; this post demonstrates for loop, while in bash scripts
On the previous post we’ve talked about parameters and options in detail and today we will talk about something is very important in shell scripting which is input & output & redirection.
On the last post, we’ve talked about input and output and redirection in bash scripting. Now you start building some Linux bash scripts, you may wonder how to run and control them on your Linux system. The only way we’ve run scripts is directly from the command line interface in real-time mode. This isn’t the only way to run Linux bash scripts in Linux.
If you get tired writing the same blocks of code over and over in your bash script. It would be nice to just write the block of code once and refer to that block of code anywhere in your bash script without having to rewrite it.
The bash shell provides a feature allowing you to do just that called Functions.
Bash functions are blocks of script code that you assign a name to and reuse anywhere in your code. Anytime you need to use that block of code in your script, you simply use the function name you assigned it.
We are going to talk about how to create your own bash functions and how to use them in other shell scripts.
On the previous post we’ve talked about bash functions and how to use it from the command line and we’ve seen some other cool stuff I recommend you to review it, Today we will talk about a very useful tool for string manipulation called sed, sed Linux command is one of the most common tools that people use to work with text files like log files, configuration files, and other text files. If you perform any type of data manipulation in your bash scripts, you want to become familiar with the sed and gawk tools in this post we are going to focus on sed Linux command and see its ability to manipulate text which is very important step in our bash scripting journey
On the previous post we’ve talked about sed Linux command and we’ve seen many examples of using it in text processing and how it is good in this, nobody can deny that sed is very handy tool but it has some limitations, sometimes you need a more advanced tool for manipulating data, one that provides a more programming-like environment giving you more control to modify data in a file more robust. This is where awk command comes in.
The awk command or GNU awk specifically because there are many extensions for awk out there takes stream editing one step further than the sed editor by providing a programming language instead of just editor commands.
In order to successfully working with the Linux sed editor and the awk command in your shell scripts you has to understand regular expressions or in short regex and to be accurate in our case it is bash regex, since there are many engines for regex you can use and we here in this regex tutorial will use the shell regex and see the bash power in working with regex.
First, we need to understand what regex is then we will dive deep into using it
In the last post, we’ve talked about regex and we see how to use them in sed and awk for text processing and we discussed before Linux sed command and awk command. During the series, we write small shell scripts but we didn’t mix things up, I think we should take a small step and write a shell script that can be some useful.
The main reason for learning to write a shell script is to be able to create your own Linux utilities. Understanding how to write useful and practical scripts is important.
However, sometimes it helps to do something fun to learn a concept or skill. The scripts in this post they can be lots of fun! And they help empower your script writing concepts.
In the previous post we’ve talked about writing practical shell scripts and we’ve seen how it is easy to write a shell script and we’ve used most of our knowledge we’ve discussed on the previous posts, today we are going to talk about a tool that does magic to our shell scripts,that tool is expect command or expect programming language. Expect command or expect programming language is a language that talks with your interactive programs or scripts that require user interaction for input. Expect works by expecting input, and upon receiving the expected input, the Expect script will send the response without any user interaction, just like magic.
The more time you spend on the Linux command line, the more you will begin to realize that you tend to use the same commands, parameters, and sequences repeatedly. Luckily, bash allows you to create aliases and and bash functions to help reduce typing
Your Own Linux..! recently published a multi-part series on Unix/Linux shell scripting, that would be useful to anyone desiring to write a bash script. The articles are:
That’s the complete list as of the publication date of this post (EDIT: Plus a couple of late additions). If you want to check to see if any have been added since then, go here.
Recent Comments