One of the things I like to standardize in firewall configs is addresses and address-sets. I always recommend creating address-sets and binding address entries into the address-set. From there, we could reference the address-set inside the security policy configuration. This makes it easier to automate changes as we no longer have to hunt inside the security policies to add or remove entries.
Recently I had a situation where I needed to parse through several thousand lines of configuration in a SRX Firewall to determine if address book entries were bound to more than one address-set. In many environments it is very likely that an address book entry is bound to more than one address-set. When that happens there are often unintended consequences that affect traffic flow through the firewall. A host may fall victim to a blocking policy; or worse yet allow more permissive access to sensitive systems!
Since I needed a quick way to do this repeatedly I wrote a simple script in python to parse through a SRX configuration. Read on to see the script!
Script
Here is the python script in its full glory (GitHub link):
#!/usr/bin/env python
import re
def extractAddress(fwConfig):
addressSplit = []
uniqueAddresses = []
for line in fwConfig:
if re.match(r'set security address-book global address ', line):
addressSplit = line.split()
uniqueAddresses.append(addressSplit[5])
return(uniqueAddresses)
def extractAddressSet(fwConfig):
addressSplit = []
addressSetDictionary = {}
for line in fwConfig:
if re.match(r'set security address-book global address-set ', line):
addressSplit = line.split()
keyValue = addressSplit[5]
addressValue = addressSplit[7]
if keyValue in addressSetDictionary:
addressSetDictionary[keyValue].append(addressValue)
else:
addressSetDictionary[keyValue] = []
addressSetDictionary[keyValue].append(addressValue)
return(addressSetDictionary)
def compareEntries(uniqueAddress, uniqueAddressSet):
for address in uniqueAddress:
tempAddressSet = []
for key, value in uniqueAddressSet.iteritems():
if address in value:
tempAddressSet.append(key)
if len(tempAddressSet) > 1:
print "Address "+address+" is in the following groups:"
print("\n".join(tempAddressSet))
def main():
addresses = []
duplicateAddresses = {}
with open (raw_input("Enter Filename: "), 'r') as fwConfig:
compare = fwConfig.readlines()
uniqueAddress = extractAddress(compare)
uniqueAddressSet = extractAddressSet(compare)
compareEntries(uniqueAddress, uniqueAddressSet)
if __name__ == "__main__":
main()
Let’s break this down step-by-step:
Modules
import re
We will be using the Regular Expression (RegEx) module. While not technically needed for a vanilla script, if you wanted to customize this for your own specific naming convention you can do so with RegEx.
Main Function
def main():
addresses = []
duplicateAddresses = {}
with open (raw_input("Enter Filename: "), 'r') as fwConfig:
compare = fwConfig.readlines()
uniqueAddress = extractAddress(compare)
uniqueAddressSet = extractAddressSet(uniqueAddress, compare)
compareEntries(uniqueAddress, uniqueAddressSet)
if __name__ == "__main__":
main()
This is the main function. In here we will open a text file (Pastebin) with the address and address-set configuration from a SRX Firewall. The output is in set mode, so make sure you use display set option when pulling the configuration from the SRX!
The file is then iterated into a variable called compare and is referenced in next two functions.
From there, we will run three functions:
extractAddress– This extracts all the addresses from the global address-book and stores them in a list calleduniqueAddressextractAddressSet– This extracts all the address-sets from the global address-book and stores them in a dictionary calleduniqueAddressSet; the address-set name is the key, and the addresses bound to that address-set are stored in a list based off the address-set keycompareEntries– This compares the addresses inuniqueAddressto the address-sets inuniqueAddressSet, and outputs the results of the check tostdout
extractAddress Function
def extractAddress(fwConfig):
addressSplit = []
uniqueAddresses = []
for line in fwConfig:
if re.match(r'set security address-book global address ', line):
addressSplit = line.split()
uniqueAddresses.append(addressSplit[5])
return(uniqueAddresses)
The extractAddress function will take the firewall configuration and iterate through it line-by-line. It is looking for the stanza set security address-book global address inside the configuration. If you didn’t notice there is an extra space at the end of the search string to ensure that we don’t accidentally extract address-sets!
If we find a line that matches the script will take the line and split it into the list addressSplit temporarily. In the set configuration a single line will be split on each space, and we can count the positions (starting from 0):
set security address-book global address AAA_SERVER_1 address 192.168.1.1/32 0 1 2 3 4 5 6 7
Since we only need the name of the address we will extract the fifth item from addressSplit and append it to the uniqueAddresses list. The uniqueAddresses list is then returned to the main function and stored as uniqueAddress.
extractAddressSet Function
def extractAddressSet(fwConfig):
addressSplit = []
addressSetDictionary = {}
for line in fwConfig:
if re.match(r'set security address-book global address-set ', line):
addressSplit = line.split()
keyValue = addressSplit[5]
addressValue = addressSplit[7]
if keyValue in addressSetDictionary:
addressSetDictionary[keyValue].append(addressValue)
else:
addressSetDictionary[keyValue] = []
addressSetDictionary[keyValue].append(addressValue)
return(addressSetDictionary)
Similar to the extractAddress function the extractAddressSet function will take the firewall configuration and iterate through it line-by-line. It is looking for the stanza set security address-book global address-set inside the configuration.
If we find a line that matches the script will take the line and split it into the list addressSplit temporarily. In the set configuration a single line will be split on each space, and we can count the positions (starting from 0):
set security address-book global address-set SERVER_GROUP_5 address AAA_SERVER_1 0 1 2 3 4 5 6 7
In this case we will need the fifth (the address-set name) and seventh value (the address that is a member of the address-set). In the dictionary addressSetDictionary we want to set the address-set name as the key, and any address that is a member of the address-set will be stored as a list under that key. We will store the fifth element as the temporary variable keyValue and the seventh element will be stored as the temporary variable addressValue.
From here, we need to determine if a key already exists in the dictionary addressSetDictionary so we will search for the keyvalue inside the dictionary:
- If the
keyvaluedoes exist we will then append theaddressValueto that key’s list. - If the
keyvaluedoes NOT exist then we will declare it as a new list inside theaddressSetDictionaryadd append theaddressValueto that list.
Once the entire configuration has been iterated the function will return the addressSetDictionary and store it as uniqueAddressSet variable.
compareEntries Function
def compareEntries(uniqueAddress, uniqueAddressSet):
for address in uniqueAddress:
tempAddressSet = []
for key, value in uniqueAddressSet.iteritems():
if address in value:
tempAddressSet.append(key)
if len(tempAddressSet) > 1:
print "Address "+address+" is in the following groups:"
print("\n".join(tempAddressSet))
This is where it all comes together. We will run a simple loop, using all the values in the uniqueAddress list to determine if there are any address matches in the uniqueAddressSet dictionary.
First, an empty list tempAddressSet is declared under the loop. This allows the list to be reset for each address in uniqueAddress. From there, we will search the dictionary uniqueAddressSet by iterating each item for any value that matches address. If a value is found, then we will store the key of the value matched in the tempAddressSet list.
After all values are iterated through we will determine if the tempAddressSet list contains more than one entry by using the len() function. If this value is greater than 1, then we will print out the name of the address as well as each address-set the address is a member of.
Proof of Concept
Attached to this post is an example configuration over at Pastebin. From there, copy the script above (or from GitHub) and save it to a file. Run the script in python and you should see an output (also on Pastebin) similar to the one below:
Clays-MBP:Desktop chaynes$ ./srx-dup.py Enter Filename: config.txt Address AAC_SERVER_3 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address AAD_SERVER_4 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address AAE_SERVER_5 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address AAF_SERVER_6 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address AAH_SERVER_8 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address AAJ_SERVER_10 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address AAK_SERVER_11 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address AAL_SERVER_12 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address AAN_SERVER_14 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address AAP_SERVER_16 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address AAQ_SERVER_17 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address AAT_SERVER_20 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address AAU_SERVER_21 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address AAW_SERVER_23 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address AAX_SERVER_24 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address AAZ_SERVER_26 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ABA_SERVER_1 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ABB_SERVER_2 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ABC_SERVER_3 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ABD_SERVER_4 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ABJ_SERVER_10 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ABK_SERVER_11 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ABL_SERVER_12 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ABN_SERVER_14 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ABP_SERVER_16 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ABQ_SERVER_17 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ABV_SERVER_22 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ABW_SERVER_23 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ABX_SERVER_24 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ABY_SERVER_25 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ABZ_SERVER_26 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ACC_SERVER_3 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ACG_SERVER_7 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ACH_SERVER_8 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ACI_SERVER_9 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ACJ_SERVER_10 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ACM_SERVER_13 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ACN_SERVER_14 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ACO_SERVER_15 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ACQ_SERVER_17 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ACR_SERVER_18 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ACU_SERVER_21 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ACW_SERVER_23 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ACX_SERVER_24 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ACY_SERVER_25 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ACZ_SERVER_26 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ADA_SERVER_1 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADB_SERVER_2 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADD_SERVER_4 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADE_SERVER_5 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ADF_SERVER_6 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ADG_SERVER_7 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADH_SERVER_8 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADI_SERVER_9 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADJ_SERVER_10 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ADK_SERVER_11 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADL_SERVER_12 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADM_SERVER_13 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADN_SERVER_14 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADO_SERVER_15 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADQ_SERVER_17 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADU_SERVER_21 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADW_SERVER_23 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADX_SERVER_24 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address ADY_SERVER_25 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address ADZ_SERVER_26 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address AEA_SERVER_1 is in the following groups: SERVER_GROUP_3 SERVER_GROUP_1 Address AEC_SERVER_3 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1 Address AEF_SERVER_6 is in the following groups: SERVER_GROUP_2 SERVER_GROUP_1
And that is it! If you have any additional feedback or improvements please let me know in the comments below!
Clay Haynes 
One comment