This article is mostly a repeat of the article How to hack the FreePBX blacklist for better call blocking capability, the only difference being that this article adds the ability to use TrueCNAM to help determine if an incoming call is from a telemarketer or robo-caller. It’s not foolproof, of course, and some spam calls may still slip through, but this will help you deal with the majority of them. BE AWARE THAT THERE MAY BE FALSE POSITIVES, where numbers that are NOT associated with telemarketers or spammers nevertheless show up with a high score. This is probably a rare occurrence, but not totally outside the realm of possibility. Therefore:

BY USING THIS METHOD YOU AGREE THAT IMPORTANT CALLS COULD BE BLOCKED. YOU ALONE ARE RESPONSIBLE FOR DETERMINING WHICH CALLS ARE BLOCKED. WE ASSUME NO RESPONSIBILITY IF YOU MISS AN IMPORTANT CALL DUE TO USE OF THIS METHOD. IF YOU DO NOT AGREE WITH THIS, DO NOT USE ANY OF THE INFORMATION IN THIS ARTICLE!!!

You will need to have an account with TrueCNAM, so go here to create one. If you are just a home user the free plan should be sufficient, particularly if you add the numbers of your most frequent callers to the Asterisk phonebook so that they are not sent to TrueCNAM for scoring (in effect the Asterisk Phonebook serves as a “whitelist” in this setup). The only “price” for this service is that you will have to submit one of your phone numbers and provide a Caller ID name to associate with it. They will call you to verify it’s your number, so make sure you can answer the verification call while you create the account.

You will also need an API Key and Password, so after you have created the account go to the API Keys page and click the “Add API Key” button to generate an API Key and Password. Keep this page open during the next steps because you will need to plug these into the script below.

Next, on your FreePBX server create a file named /var/lib/asterisk/agi-bin/truecnam.agi, and change the ownership and permissions so that Asterisk can use it (you will need to be root to do this, or use sudo in front of each of the following commands):

touch /var/lib/asterisk/agi-bin/truecnam.agi
chown asterisk:asterisk /var/lib/asterisk/agi-bin/truecnam.agi
chmod 754 /var/lib/asterisk/agi-bin/truecnam.agi

Next use a text editor such as nano to open the file:

nano /var/lib/asterisk/agi-bin/truecnam.agi

Paste in the following lines, changing the API Key and Password values to the ones shown on the API Keys page:

#!/bin/bash
declare -a array
while read -e ARG && [ "$ARG" ] ; do
        array=(` echo $ARG | sed -e 's/://'`)
        export ${array[0]}=${array[1]}
done

APIKEY="APIKey"
APIPASS="APIPassword"
RESULT=""
SPAMSCORE=0
RESULT=$(curl -s -k -m 2 --connect-timeout 2 "https://api.truecnam.net/api/v1?username=$APIKEY&password=$APIPASS&resp_type=extended&resp_format=csv&calling_number=$agi_callerid&call_party=terminating")
SPAMSCORE=`echo $RESULT | cut -d , -f 9`

if [[ $SPAMSCORE == "" ]]
then
  SPAMSCORE=0
fi

echo "SET VARIABLE SPAMSCORE ${SPAMSCORE}"

(The above script was modified from scripts posted in this thread at DSLReports)

In case you missed it above, THIS SCRIPT WILL NOT WORK UNLESS YOU PASTE IN YOUR API KEY AND PASSWORD IN THE APPROPRIATE LOCATIONS.

What follows next is the original article How to hack the FreePBX blacklist for better call blocking capability, but with the modifications to use the above script:

Have you ever wished that you could blacklist calls in FreePBX using criteria other than just the exact phone number? It’s quite possible, and here’s an explanation of how it can be accomplished.

First, using a text editor such as nano, go into the file /etc/asterisk/extensions_additional.conf and find the context headed [app-blacklist-check]. It will probably look like this:

[app-blacklist-check] include => app-blacklist-check-custom
exten => s,1(check),GotoIf($["${BLACKLIST()}"="1"]?blacklisted)
exten => s,n,Set(CALLED_BLACKLIST=1)
exten => s,n,Return()
exten => s,n(blacklisted),Answer
exten => s,n,Wait(1)
exten => s,n,Zapateller()
exten => s,n,Playback(ss-noservice)
exten => s,n,Hangup

Note that this is a pretty simple macro context. The first line checks to see if a caller ID number appears in the Asterisk blacklist, and if so, it shuffles the call off to a “number not in service” recording.

If you copy and paste this entire context into the file /etc/asterisk/extensions_override_freepbx.conf and remove the “include” line at the start of the macro, you can then modify it to do what you want. Let’s look at that same context again, but with a few added lines, as it might appear in /etc/asterisk/extensions_override_freepbx.conf

[app-blacklist-check]

;Make sure CDR DID field is set on blacklisted calls
exten => s,1(check),Set(CDR(did)=${FROM_DID})
;Next line is from the original blacklist context but moved higher
exten => s,n,Set(CALLED_BLACKLIST=1)
;Special CallerID name lookup; uses Asterisk Phonebook which also acts as whitelist
exten => s,n,GotoIf($["${DB_EXISTS(cidname/${CALLERID(num)})}"!="1"]?blcheck)
exten => s,n,Set(CALLERID(name)=${DB(cidname/${CALLERID(num)})})
exten => s,n,Set(__SKIPCIDLOOKUP=1)
exten => s,n,Return()
;Original first line of context with line number changed and blcheck label added
exten => s,n(blcheck),GotoIf($["${BLACKLIST()}"="1"]?blacklisted)
;Blacklist Caller ID names found in custom Asterisk database blistname if used
exten => s,n,GotoIf($["${DB_EXISTS(blistname/${TOUPPER(${CALLERID(name)})})}"="1"]?blacklisted)
;Blacklist Caller ID names of single letter followed by 9 or more digits
exten => s,n,Set(regx=^[A-Z][0-9]{9}[0-9]* *$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${CALLERID(name)})} = 1]?blacklisted)
;Blacklist numbers where first digit of 7 digit local number is 0 or 1
exten => s,n,Set(regx=^[+]?1?[0-9]{3}[01][0-9]{6}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${CALLERID(num)})} = 1]?blacklisted)
;Blacklist numbers where first digit of area code is 0 or 1
exten => s,n,Set(regx=^[+]?1?[01][0-9]{9}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${CALLERID(num)})} = 1]?blacklisted)
;Blacklist numbers where first three digits of 7 digit local number are 555
exten => s,n,Set(regx=^[+]?1?[0-9]{3}555[0-9]{4}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${CALLERID(num)})} = 1]?blacklisted)
;Blacklist numbers that start with 1 but are longer than 11 digits total
exten => s,n,Set(regx=^[+]?1[0-9]{11}[0-9]*$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${CALLERID(num)})} = 1]?blacklisted)
;Blacklist numbers of less than 7 digits in length
exten => s,n,GotoIf($[${LEN(${CALLERID(num)})} < 7]?blacklisted)
;Blacklist numbers of 8 digits in length
exten => s,n,GotoIf($[${LEN(${CALLERID(num)})} = 8]?blacklisted)
;Query TrueCNAM
exten => s,n,AGI(truecnam.agi)
exten => s,n,Noop(TrueCNAM SPAMSCORE: ${SPAMSCORE})
;Show non-zero spam score in Userfield of Call Detail Record
exten => s,n,ExecIf($[${SPAMSCORE} > 0]?Set(CDR(userfield)="Spam Score ${SPAMSCORE}"))
exten => s,n,ExecIf($[${SPAMSCORE} > 39]?Set(CALLERID(name)=SPAM:${CALLERID(name)}))
exten => s,n,GotoIf($[${SPAMSCORE} > 99]?highscore)
;More original blacklist context
exten => s,n,Return()
;Set Call Detail Record Userfield to Blacklisted (splits statement from original blacklist context)
exten => s,n(blacklisted),Set(CDR(userfield)=Blacklisted)
exten => s,n(highscore),Answer
;More original blacklist context
exten => s,n,Wait(1)
exten => s,n,Zapateller()
exten => s,n,Playback(ss-noservice)
exten => s,n,Hangup

;--== end of [app-blacklist-check] ==--;

The lines in blue are from the original context that was copied from /etc/asterisk/extensions_additional.conf. We should mention here that not all the ideas in the new lines are original with us; some are adaptations of things we’ve read over many months in various forums.

EDIT: Also please note that in recent versions of FreePBX the last four dialplan lines have been changed. Instead of

exten => s,n,Wait(1)
exten => s,n,Zapateller()
exten => s,n,Playback(ss-noservice)
exten => s,n,Hangup

It now uses

exten => s,n,Set(BLDEST=${DB(blacklist/dest)})
exten => s,n,GotoIf(${LEN(${BLDEST})}?${BLDEST}:app-blackhole,zapateller,1)

Or in even newer versions:

exten => s,n,Set(BLDEST=${DB(blacklist/dest)})
exten => s,n,GotoIf($["${returnhere}"="1"]?returnto)
exten => s,n,GotoIf(${LEN(${BLDEST})}?${BLDEST}:app-blackhole,zapateller,1)
exten => s,n(returnto),Return()

This underscores the importance of starting with the original [app-blacklist-check] context in your FreePBX application, and not just blindly copying the example dialplan shown here. (End of edit.)

You don’t need to use all of these added lines shown. On the other hand, you can add your own customizations. Here is what each added line (or set of lines) means:

The change to line 1 (moving the original line 1 down a few lines) adds Set(CDR(did)=${FROM_DID}), which causes the call detail record to show the DID the call came in on for blacklisted calls.

The next added section consists of these lines:

;Special CallerID name lookup; uses Asterisk Phonebook which also acts as whitelist
exten => s,n,GotoIf($["${DB_EXISTS(cidname/${CALLERID(num)})}"!="1"]?blcheck)
exten => s,n,Set(CALLERID(name)=${DB(cidname/${CALLERID(num)})})
exten => s,n,Set(__SKIPCIDLOOKUP=1)
exten => s,n,Return()

What this does is checks the Asterisk Phonebook to see if the calling number appears there, and if it does, it sets the Caller ID name to the name associated with that number in the Phonebook. It also allows the Asterisk Phonebook to act as a “whitelist”, so that numbers in the Phonebook are not sent to TrueCNAM for spam scoring or subjected to any further blacklist tests. If a number appears in both the Asterisk Phonebook and the Blacklist, the Asterisk Phonebook will “win” and the call will be allowed through, so don’t put any number from which you want to block calls in the Asterisk Phonebook! Originally I had put this section lower in the context but then I realized that if a number appears in the whitelist, no further blacklist checking is necessary or desirable.

There is one issue with doing this, and that is that if you have the FreePBX CallerID Lookup module installed and enabled, and you use another Caller ID lookup source it will override the Asterisk Phonebook lookup. To avoid that, you can either uninstall or remove the CallerID Lookup module, OR you can copy the [cidlookup] context from /etc/asterisk/extensions_additional.conf to /etc/asterisk/extensions_override_freepbx.conf and change the first line to a test for the SKIPCIDLOOKUP variable. For example, if the first line is:

exten => cidlookup_return,1,ExecIf($["${DB(cidname/${CALLERID(num)})}" != ""]?Set(CALLERID(name)=${DB(cidname/${CALLERID(num)})}))

You could change it to:

exten => cidlookup_return,1,,ExecIf($["${SKIPCIDLOOKUP}"="1"]?Return())
exten => cidlookup_return,n,ExecIf($["${DB(cidname/${CALLERID(num)})}" != ""]?Set(CALLERID(name)=${DB(cidname/${CALLERID(num)})}))

Don’t forget to change the line number 1 to n in the original first line, since it’s no longer line 1.  If you do not have the [cidlookup] context in /etc/asterisk/extensions_additional.conf then you probably have not installed or enabled the CallerID Lookup module, and in that case you don’t need to make these changes, and you can also omit the line that contains exten => s,n,Set(__SKIPCIDLOOKUP=1) from the added section under discussion.

Also, if you have the CID Superfecta module installed and enabled, then you should not enable the Asterisk Phonebook data source (since you would be doing the same lookup twice), but you should enable the Trunk Provided data source (so that CID Superfecta will see that there is already a Caller ID name associated with the call, and not further attempt to find a name to associate with the number).

Getting back to the additions to the app-blacklist-check context…

;Blacklist Caller ID names found in custom Asterisk database blistname if used
exten => s,n,GotoIf($["${DB_EXISTS(blistname/${TOUPPER(${CALLERID(name)})})}"="1"]?blacklisted)

This addition allows you to blacklist by exact Caller ID name. To add a name to the blacklist-by-name database, go into the Asterisk CLI and enter this:

database put blistname NAME 1

Where NAME is the name you want to add – enclose it in quotation marks if it contains spaces:

database put blistname "SOME NAME" 1

You can use a descriptive word if you like in place of the number 1 at the end of the line. To delete an already entered name:

database del blistname NAME

And to see what’s already in the blacklist-by-name database:

database show blistname

Next we come to:

;Blacklist Caller ID names of single letter followed by 9 or more digits
exten => s,n,Set(regx=^[A-Z][0-9]{9}[0-9]* *$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${CALLERID(name)})} = 1]?blacklisted)

These lines are an example of blacklisting a Caller ID name using a regular expression. This will catch any Caller ID name that starts with a single letter and is followed by 9 or more digits, and any number of spaces at the end. This particular example will get rid of those irritating Vnnnnnnnnn… name calls, even if they change the starting letter to something other than “V”.

;Blacklist numbers where first digit of 7 digit local number is 0 or 1
exten => s,n,Set(regx=^[+]?1?[0-9]{3}[01][0-9]{6}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${CALLERID(num)})} = 1]?blacklisted)

;Blacklist numbers where first digit of area code is 0 or 1
exten => s,n,Set(regx=^[+]?1?[01][0-9]{9}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${CALLERID(num)})} = 1]?blacklisted)

;Blacklist numbers where first three digits of 7 digit local number are 555
exten => s,n,Set(regx=^[+]?1?[0-9]{3}555[0-9]{4}$)
exten => s,n,GotoIf($[${REGEX("${regx}" ${CALLERID(num)})} = 1]?blacklisted)

These are three examples of blacklisting numbers by regular expression patterns. The first example blacklists numbers that have an invalid exchange (where the 7-digit part of the number starts with 0 or 1), while the second blacklists numbers that come in with an area code starting with 0 or 1, which is invalid. The third blocks numbers that appear to come from the “directory assistance” exchange 555 in any area code. In all cases, it does not matter if the number starts with a leading + and/or 1. Note that Asterisk has a fairly restrictive syntax for regular expressions, so you can’t do many of the substitutions that would be possible in other computer languages. See the article, Did you know that Asterisk has the ability to evaluate Regular Expressions, though not in the same way as Perl or FreeSWITCH? for more information.

You can always add additional blacklist patterns by duplicating an appropriate pair of lines (for blacklisting by name or number pattern) and then modifying the regular expression. For example, to block calls from certain specific area codes, you could use a regular expression of the form ^[+]?1?(xxx|yyy|zzz)[0-9]{7}$ where xxx, yyy, and zzz are the area codes you want to block. You can include as many area codes as you need, separated by the | character.

;Blacklist numbers of 8 digits in length
exten => s,n,GotoIf($[${LEN(${CALLERID(num)})} = 8]?blacklisted)

This is an example of blocking numbers of a specific length. If you have a bad caller that sends numbers that are exactly 8 characters in length, this will catch them. You can, of course, change the “8” to catch other bad length numbers, but be careful not to inadvertently block wanted international or other valid calls with non-standard number lengths.

The next section is where the TrueCNAM scoring is performed:

;Query TrueCNAM
exten => s,n,AGI(truecnam.agi)
exten => s,n,Noop(TrueCNAM SPAMSCORE: ${SPAMSCORE})

The above calls the bash script to get the TrueCNAM spam score for the number (from 0 to 100).

;Show non-zero spam score in Userfield of Call Detail Record
exten => s,n,ExecIf($[${SPAMSCORE} > 0]?Set(CDR(userfield)="Spam Score ${SPAMSCORE}"))

Newer versions of FreePBX have a “Userfield” in the call detail records (CDR), and here we are placing the spam score (if it is higher than zero) in the CDR so that when you review the call detail you can determine how a call scored.  For example if a calling number had a relatively high score, but not quite high enough to get it blocked, you could review it using an online lookup source such as 800notes to see if it’s a number you want to add to the Blacklist.

exten => s,n,ExecIf($[${SPAMSCORE} > 39]?Set(CALLERID(name)=SPAM:${CALLERID(name)}))
exten => s,n,GotoIf($[${SPAMSCORE} > 99]?highscore)
exten => s,n,Return()

The above section is where we act on the spam score. In this example, if the spam score is 40 or higher, the word “SPAM:” is prepended to the existing caller ID name.  This does not mean that the call is unequivocally from a telemarketer or robocaller, but that you probably should be suspicious of that caller.  You can change the value but be careful, because in our experience calls from some perfectly valid numbers can have a score of 30 (it’s rare but it happens).  If the spam score is 100 (the highest possible score), then it basically treats the call as if you had blacklisted it.  You can change this to a lower value (TrueCNAM recommends 80 or higher, so you could change > 99 to > 79) but that is entirely up to you.  We didn’t want anyone blaming us for missing that all important call that would have awarded them the vacation of a lifetime (but only after sending gift cards to some Nigerian prince to cover “fees”), so in this example we set it to only block the call outright if it scores a solid 100.

;Set Call Detail Record Userfield to Blacklisted (splits statement from original blacklist context)
exten => s,n(blacklisted),Set(CDR(userfield)=Blacklisted)

This is where calls that fail the other tests (not TrueCNAM) land. We don’t recall where we first saw this, but it is a great idea. Here the “Userfield” in the call detail record is set to the word “Blacklisted”. When you are looking at your call records, this tells you instantly if a problem caller is already being blacklisted, so you don’t need to wonder if you should add them to the blacklist. Note that if you see a Spam Score, the call is NOT blacklisted, so if you want to blacklist that number you will need to do that separately. You might want to do that for a repeat problem caller, so you’re not reaching your TrueCNAM daily limit prematurely by looking up what you already know is a bad caller.

exten => s,n(highscore),Answer

The above line is where calls that scored above the “highscore” limit in TrueCNAM land. So they are treated like any other blacklisted call, except the spam score appears in the CDR “Userfield” instead of the word “Blacklisted”.

Keep in mind that any time you replace a context in /etc/asterisk/extensions_override_freepbx.conf, it is your responsibility to make sure it “tracks” any changes to the original context in /etc/asterisk/extensions_additional.conf. So, after any major FreePBX upgrade, you should take a look at the original [app-blacklist-check] context (and the [cidlookup] context, if you have modified that) and make sure that they have not changed, and if they have, that you make the same changes to your replacement contexts.

One additional point – at the top of each of the original contexts, there is an “include” statement that we suggest you remove when you copy the context to /etc/asterisk/extensions_override_freepbx.conf, for example, “include => app-blacklist-check-custom“. You might wonder if you could create a context named app-blacklist-check-custom in /etc/asterisk/extensions_custom.conf and copy the original context there. But, for whatever reason, that just doesn’t seem to work, and there’s no real advantage to putting the copied context there anyway.

Related articles:
How to hack the FreePBX blacklist for better call blocking capability
Link: Asterisk: Blacklisting For Multiple Users
Forum thread: User-specific Caller ID/whitelist/blacklist in Asterisk/FreePBX