I wrote a script a few years ago that would check to see if address entries existed in the more than one address-set. It works well, but it was missing one key component – what happens if an address-set entry contains another address-set?

I’ve since updated the script with a little bit of logic to include tracking this very scenario, and I’ll explain the changes below. The script is still available on GitHub if you want to download a copy and use it for your operations!

Script Updates

Here is the python script in its full glory (GitHub link):

#!/usr/bin/env python

import re

def extractAddress(fwConfig):
    addressSplit = []
    uniqueAddresses = []

    for line in fwConfig:
        if re.match(r'set security address-book global address ', line):

            addressSplit = line.split()
            uniqueAddresses.append(addressSplit[5])

    return(uniqueAddresses)

def extractAddressSetFromAddressSet(fwConfig):
    addressSplit = []
    addressSetFromAddressSet = {}

    for line in fwConfig:
        if re.match(r'set security address-book global address-set .* address-set', line):
            addressSplit = line.split()

            keyValue = addressSplit[5]
            addressValue = addressSplit[7]

            if keyValue in addressSetFromAddressSet:
                addressSetFromAddressSet[keyValue].append(addressValue)

            else:
                addressSetFromAddressSet[keyValue] = []
                addressSetFromAddressSet[keyValue].append(addressValue)

    return(addressSetFromAddressSet)

def extractAddressSet(fwConfig):
    addressSplit = []
    addressSetDictionary = {}

    for line in fwConfig:
        if re.match(r'set security address-book global address-set ', line):
            addressSplit = line.split()

            keyValue = addressSplit[5]
            addressValue = addressSplit[7]

            if keyValue in addressSetDictionary:
                addressSetDictionary[keyValue].append(addressValue)

            else:
                addressSetDictionary[keyValue] = []
                addressSetDictionary[keyValue].append(addressValue)

    return(addressSetDictionary)

def compareEntries(uniqueAddress, uniqueAddressSet, addressSetFromAddressSet):

    for key, value in addressSetFromAddressSet.items():
        for item in range(len(value)):
            addressSetCompare = value[item]
            if addressSetCompare in uniqueAddressSet:
                for addressEntry in uniqueAddressSet[addressSetCompare]:
                    uniqueAddressSet[key].append(addressEntry)

    for address in uniqueAddress:
        tempAddressSet = []
        for key, value in uniqueAddressSet.iteritems():
            if address in value:
                tempAddressSet.append(key)
        if len(tempAddressSet) > 1:
            print "Address "+address+" is in the following groups:"
            print("\n".join(tempAddressSet))

def main():
    addresses = []
    duplicateAddresses = {}
    with open (raw_input("Enter Filename: "), 'r') as fwConfig:

        compare = fwConfig.readlines()
        uniqueAddress = extractAddress(compare)
        addressSetFromAddressSet = extractAddressSetFromAddressSet(compare)
        uniqueAddressSet = extractAddressSet(compare)
        compareEntries(uniqueAddress, uniqueAddressSet, addressSetFromAddressSet)

if __name__ == "__main__":
    main()

Let’s break this down the changes step-by-step:

Main Function

def main():
    addresses = []
    duplicateAddresses = {}
    with open (raw_input("Enter Filename: "), 'r') as fwConfig:

        compare = fwConfig.readlines()
        uniqueAddress = extractAddress(compare)
        addressSetFromAddressSet = extractAddressSetFromAddressSet(compare)
        uniqueAddressSet = extractAddressSet(compare)
        compareEntries(uniqueAddress, uniqueAddressSet, addressSetFromAddressSet)


if __name__ == "__main__":
    main()

In the main function I’ve modified it a bit, and now extract out all address-sets which contain address-sets as members as a new function.

This is the new function:

  • extractAddressSetFromAddressSet – This extracts all the addresses-set children from a root address-set in a dictionary called addressSetFromAddressSet; the address-set parent’s name is the key, and the address-set children bound to that address-set are stored in a list based off the parent address-set key

We have also modified the compareEntries function to include the new addressSetFromAddressSet variable.

extractAddressSetFromAddressSet Function

def extractAddressSetFromAddressSet(fwConfig):
    addressSplit = []
    addressSetFromAddressSet = {}

    for line in fwConfig:
        if re.match(r'set security address-book global address-set .* address-set', line):
            addressSplit = line.split()

            keyValue = addressSplit[5]
            addressValue = addressSplit[7]

            if keyValue in addressSetFromAddressSet:
                addressSetFromAddressSet[keyValue].append(addressValue)

            else:
                addressSetFromAddressSet[keyValue] = []
                addressSetFromAddressSet[keyValue].append(addressValue)

    return(addressSetFromAddressSet)

The extractAddressSetFromAddressSet function will take the firewall configuration and iterate through it line-by-line. It is looking for the stanza set security address-book global address-set .* address-set  inside the configuration.

If we find a line that matches the script will take the line and split it into the list addressSplit temporarily. In the set configuration a single line will be split on each space, and we can count the positions (starting from 0):

set security address-book global address-set ALL_AAA_SERVERS address-set AAA_SERVER_1
0   1        2            3      4       5            6       7

Since we only need the name of the address we will extract the fifth item from addressSplit and append it to the uniqueAddresses list. The uniqueAddresses list is then returned to the main function and stored as uniqueAddress.

compareEntries Function

def compareEntries(uniqueAddress, uniqueAddressSet, addressSetFromAddressSet):

    for key, value in addressSetFromAddressSet.items():
        for item in range(len(value)):
            addressSetCompare = value[item]
            if addressSetCompare in uniqueAddressSet:
                for addressEntry in uniqueAddressSet[addressSetCompare]:
                    uniqueAddressSet[key].append(addressEntry)

    for address in uniqueAddress:
        tempAddressSet = []
        for key, value in uniqueAddressSet.iteritems():
            if address in value:
                tempAddressSet.append(key)
        if len(tempAddressSet) > 1:
            print "Address "+address+" is in the following groups:"
            print("\n".join(tempAddressSet))

As shown in the snippet above, the addressSetFromAddressSet dictionary is processed first. Let’s first see how our data looks right now:

Data Diagram

We can ignore the uniqueAddress data as we won’t be using it for the first for loop. We first extract the key and value from addressSetFromAddressSet – the key in this case is the parent address-set entry, and the value is a list of all child address-sets from that key value. Since we need to iterate each child address-set, we’ll use a for loop to do so; iterating each item in the list. A temporary variable called addressSetCompare stores the item as a string. From there, the addressSetCompare value is compared to all key values inside the uniqueAddressSet dictionary, and if one matches we will then iterate through each addressEntry value belonging to that key inside the uniqueAddressSet dictionary, and append it to the original key value at the start of the function.

The rest of the compare function stays the same, and runs after we expand the address-sets.

Proof of Concept

I’ve updated the Pastebin PoC configuration. From there, copy the script above (or from GitHub) and save it to a file. Run the script in python and you should see an output (also on Pastebin) similar to the one below:

Clays-MBP:Desktop chaynes$ ./srx-dup.py 
Enter Filename: config.txt

Address AAA_SERVER_1 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AAB_SERVER_2 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AAC_SERVER_3 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address AAD_SERVER_4 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address AAE_SERVER_5 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address AAF_SERVER_6 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address AAG_SERVER_7 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AAH_SERVER_8 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address AAI_SERVER_9 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AAJ_SERVER_10 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address AAK_SERVER_11 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address AAL_SERVER_12 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address AAM_SERVER_13 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AAN_SERVER_14 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address AAO_SERVER_15 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AAP_SERVER_16 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address AAQ_SERVER_17 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address AAR_SERVER_18 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AAS_SERVER_19 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AAT_SERVER_20 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address AAU_SERVER_21 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address AAV_SERVER_22 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AAW_SERVER_23 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address AAX_SERVER_24 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address AAY_SERVER_25 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AAZ_SERVER_26 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ABA_SERVER_1 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ABB_SERVER_2 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ABC_SERVER_3 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ABD_SERVER_4 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ABE_SERVER_5 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABF_SERVER_6 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABG_SERVER_7 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABH_SERVER_8 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABI_SERVER_9 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABJ_SERVER_10 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ABK_SERVER_11 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ABL_SERVER_12 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ABM_SERVER_13 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABN_SERVER_14 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ABO_SERVER_15 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABP_SERVER_16 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ABQ_SERVER_17 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ABR_SERVER_18 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABS_SERVER_19 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABT_SERVER_20 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABU_SERVER_21 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ABV_SERVER_22 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ABW_SERVER_23 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ABX_SERVER_24 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ABY_SERVER_25 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ABZ_SERVER_26 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ACA_SERVER_1 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACB_SERVER_2 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACC_SERVER_3 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ACD_SERVER_4 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACE_SERVER_5 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACF_SERVER_6 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACG_SERVER_7 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ACH_SERVER_8 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ACI_SERVER_9 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ACJ_SERVER_10 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ACK_SERVER_11 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACL_SERVER_12 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACM_SERVER_13 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ACN_SERVER_14 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ACO_SERVER_15 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ACP_SERVER_16 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACQ_SERVER_17 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ACR_SERVER_18 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ACS_SERVER_19 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACT_SERVER_20 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACU_SERVER_21 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ACV_SERVER_22 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ACW_SERVER_23 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ACX_SERVER_24 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ACY_SERVER_25 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ACZ_SERVER_26 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ADA_SERVER_1 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADB_SERVER_2 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADC_SERVER_3 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ADD_SERVER_4 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADE_SERVER_5 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ADF_SERVER_6 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ADG_SERVER_7 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADH_SERVER_8 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADI_SERVER_9 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADJ_SERVER_10 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ADK_SERVER_11 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADL_SERVER_12 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADM_SERVER_13 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADN_SERVER_14 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADO_SERVER_15 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADP_SERVER_16 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ADQ_SERVER_17 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADR_SERVER_18 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ADS_SERVER_19 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ADT_SERVER_20 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ADU_SERVER_21 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADV_SERVER_22 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address ADW_SERVER_23 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADX_SERVER_24 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address ADY_SERVER_25 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address ADZ_SERVER_26 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address AEA_SERVER_1 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_3
SERVER_GROUP_1
Address AEB_SERVER_2 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AEC_SERVER_3 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1
Address AED_SERVER_4 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AEE_SERVER_5 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_1
Address AEF_SERVER_6 is in the following groups:
SERVER_GROUP_4
SERVER_GROUP_2
SERVER_GROUP_1

And that is it! If you have any additional feedback or improvements please let me know in the comments below!

Tags: , , , ,

Clay is a recovering server guy that has learned how networks and security worked and used it to his advantage. He is always eager to learn a new technology or technique as there are never enough tools in the toolbox for solving problems in modern networking and server environment. Clay is always striving to make the "more perfect" network available to peers and users. He currently an active Juniper Ambassador, and has contributed to multiple Juniper Day One guides to solve both old and new problems that engineers face in their networks. Clay is a talented individual that enjoys the new challenges that are facing next generation networks, both large and small.

Clay has achieved two of the highest certifications from Juniper Networks: the JNCIE-SEC (Certification Number #69) for his proficiency in Juniper Networks security platforms, and the JNCIE-ENT (Certification Number #492) for his proficiency in Juniper Networks enterprise routing and switching platforms. He is also certified as a Pulse Secure Certified Instructor and Palo Alto Networks Certified Network Security Instructor. Outside of the of the certifications above, Clay also has experience with securing cloud-provider networks while ensuring the highest levels of performance to serve its users.

Leave a Reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.