Working with Dictionaries and using pyATS DQ

The first thing when using the inbuilt Python functions are that most do not have networking specific examples. Sometimes I find it confusing to look at the vanilla examples because I can’t relate the content back to a specific networking example. In this post, I’ll review a couple of options around extracting data from a dictionary object.

Firstly, Python Dictionary Comprehensions are really useful for making easy work of pulling out data from for/loops and also allow conditions to be applied at the same time. What you’re saving here is the amount of code to write where a single line can replace multiple lines.

Let’s start with an example dataset which is a dictionary. Let’s imagine we pulled this from a device.

vrf_info = {'VRF' : 'DC-LAN', 'RD' : '65002:23', 'Route-Target-Import': '65002:123', 'Route-Target-Export': '65002:123', 'description' : 'Datacenter LAN'}

Let’s imagine we wanted to pull off information from this dictionary. Let’s first see if our vrf_info variable contains a value with “DC-LAN”. Of course we can see it’s there but let’s run a query to see if it’s there and map the result into a new variable called vrf_output

vrf_output  = {k:v for (k,v) in vrf_info.items() if v=='DC-LAN'}

>>> vrf_output
{'VRF': 'DC-LAN'}
>>> type(vrf_output)
<class 'dict'>

We know we’re working with a dictionary with the curly braces. k:v is the key and value mapping container for the loop. We have a for loop iterating over the vrf_info dictionary (this is why .items() is needed as that is the entire contents of the dictionary. ) The “v” is a value from the key. When the loop reaches the ‘VRF’ key which is represented in the loop by “k” then the value of “v” will be ‘DC-LAN’ this will cause the condition to be met. Python will reconstruct a new dictionary which contains only the matching data. We can see that from printing the vrf_output variable and checking it’s type.

Some further examples:

import re

vrf_output  = {k:v for (k,v) in vrf_info.items() if re.findall(r'^65.',v)} 
>>> vrf_output
{'RD': '65002:23', 'Route-Target-Import': '65002:123', 'Route-Target-Export': '65002:123'}
 

Here we are using a regex expression and findall to look for any values which begin with “65” which matches the Route-Distinguisher, Route-Target-Import and Route-Target-Export and creates a new dictionary with the matching values.

vrf_output  = {k:v for (k,v) in vrf_info.items() if k=='Route-Target-Import' and v.startswith('65')} 

>>> vrf_output
{'Route-Target-Import': '65002:123'}

Here we look over that dataset to match the key ‘Route-Target-Import’ and then we check the value starts with ’65’ which it does of course. The traditional way of doing this is through a for loop and then an if statement. Finally you would need another line to assign the value to the variable whereas this is all contained in a single line. It’s possible to do this for lists as well.

If you’re finding this a bit heavy going then I suggest using Dq in pyATS which can do similar things. I would recommend using pyATS Dq as the most flexible way to obtain the data.

Let’s have a little play with Dq and use that to grab information for with small set of data from a dictionary.

vrf_info ={'R1': {'VRF' : 'DC-LAN', 'RD' : '65002:23', 'Route-Target-Import': '65002:123', 'Route-Target-Export': '65002:123', 'description' : 'Datacenter LAN'}}

Let’s say we have another router “R2” which has the same dataset which needs to be added into our main dictionary which is called vrf_info. We’ve mapped that data into a variable called vrf_add here. We then use the update function to combine those two datasets together.

vrf_add={'R2': {'VRF': 'CORP-LAN', 'RD': '64001:23', 'Route-Target-Import': '64002:123', 'Route-Target-Export': '64002:123', 'description': 'Corporate LAN'}}

vrf_info.update(vrf_add)

This will give us a combined dictionary with the two entries for R1 and R2. (see below).

{'R1': {'VRF': 'DC-LAN', 'RD': '65002:23', 'Route-Target-Import': '65002:123', 'Route-Target-Export': '65002:123', 'description': 'Datacenter LAN'}, 'R2': {'VRF': 'CORP-LAN', 'RD': '64001:23', 'Route-Target-Import': '64002:123', 'Route-Target-Export': '64002:123', 'description': 'Corporate LAN'}}

Now let’s start writing queries using Dq to pull out information or records from that data set. First one, we looking for any Routers which have a RD value of 64001:23. The second one is the inverse in not matching those values. The reconstruct() will create a new dictionary from those values.

Dq(vrf_info).contains_key_value('RD', '64001:23').reconstruct()

{'R2': {'RD': '64001:23'}}

Dq(vrf_info).not_contains_key_value('RD', '64001:23').reconstruct()

{'R1': {'VRF': 'DC-LAN', 'RD': '65002:23', 'Route-Target-Import': '65002:123', 'Route-Target-Export': '65002:123', 'description': 'Datacenter LAN'}, 'R2': {'VRF': 'CORP-LAN', 'Route-Target-Import': '64002:123', 'Route-Target-Export': '64002:123', 'description': 'Corporate LAN'}}

We can do more complex queries using the power of REGEX. Here we match the RD Value which starts with “65002”

Dq(vrf_info).contains_key_value('RD', '65002.*', value_regex=True).reconstruct()

{'R1': {'RD': '65002:23'}}

We can find all the routers which have a description which contains the word “Corp” to find the Corporate LAN.

Dq(vrf_info).contains_key_value('description', 'Corp.*', value_regex=True).reconstruct()

{'R2': {'description': 'Corporate LAN'}}

Dq(vrf_info).contains_key_value('description', '.*LAN.*', value_regex=True).reconstruct()

{'R1': {'description': 'Datacenter LAN'}, 'R2': {'description': 'Corporate LAN'}}

We can use contains to pull out all the relevant values for a specific key in this example the route-targets values which are being exported.

Dq(vrf_info).contains('Route-Target-Export').reconstruct()

{'R1': {'Route-Target-Export': '65002:123'}, 'R2': {'Route-Target-Export': '64002:123'}}

Hopefully this has given a brief overview of how information can be extracted from a dictionary. Clearly the use case will be to query output coming from the CLI for certain conditions e.g. all the BGP sessions which are not established, all interfaces in a down state and so on. I think it’s very important that you grasp how the data is queried within dictionaries and lists. It’s going to be essential to any scripts which you write which check the operational state of the network or for auditing the network.