Stage Two – Walk-through of the Parser script
Having setup the environment then the first thing we need to explore is the structure of the files and the directories. For this example, I will be writing a parser for show ip nbar protocol discovery
Command extract:
R1#sh ip nbar protocol-discovery protocol
GigabitEthernet1
Last clearing of "show ip nbar protocol-discovery" counters 00:13:45
Input Output
----- ------
Protocol Packet Count Packet Count
Byte Count Byte Count
5min Bit Rate (bps) 5min Bit Rate (bps)
5min Max Bit Rate (bps) 5min Max Bit Rate (bps)
---------------------------- ------------------------ ------------------------
ssh 191 134
24805 22072
2000 1000
1999 1001
unknown 172 503
39713 31378
0 0
3000 0
ping 144 144
14592 14592
0 0
1000 1000
dns 107 0
21149 0
0 0
2000 0
vrrp 0 738
0 39852
0 0
0 0
ldp 174 175
13224 13300
0 0
0 0
ospf 86 87
9460 9570
0 0
0 0
Total 874 1781
122943 130764
2000 1000
8000 2000 The first thing of note is to know where the existing parsers are stored. Since my command is IOS-XE then I need to navigate to :
genieparser/src/genie/libs/parser/iosxe
What you find in that directory is the list of existing parsers. Naming standards seen to suggest that I should call my file show_nbar.py which is my interpretation of the standard naming used for the files. However, I don’t know if there is an actual documented standard to follow in terms of naming the file. As usual, the obvious thing is to use something which aligns with previous ones. Create your empty file and save it into the directory. There are more files to consider later around the unit testing and change log but I’ll park those for later.
Let’s start by putting the hierarchy into our show_nbar.py script.
Use the make json command at the point when you are ready to test/debug your parser. Make sure you are in the genieparser directory where the makefile is located.
The starting point will be with the module which will be imported. This I think is fairly standard bolierplate module imports.
The genie.metaparser.util.schemaengine is used for the schema and the imports on the end (e.g. Any, Or, Optional). All the imports may or may not all be required. However, it’s easier to include them for now and they will make more sense when we start to look at the schema in more detail later.
from genie.metaparser import MetaParser
from genie.metaparser.util.schemaengine import Any, Or, Optional
import reSpeaking of the schema class it’s the first thing we define. This comment field is presentable and should be defined to explain what command we are using.
# ======================================================
# Schema for 'show ip nbar protocol-discovery protocol'
# ======================================================
class ShowIpNbarDiscoverySchema(MetaParser):
schema = {}We have called the schema class ShowIpNbarDiscoverySchema and the naming standard should broadly follow this. I am not sure how rigid that is but it makes sense to replicate (including the first letter capitalization).
I am not going to fill in the schema class yet because I believe the process should be define what comes out of the dictionary first before defining the schema around it. That is why I have defined an empty one {} at this stage. You may want to define a schema first if that is your personal preference. If so then please go to the section where we look at the schema class in more detail.
Next we are looking at the main part of the parser where we return a dictionary from all the REGEX patterns that we define. Again, we need to define a class name and to keep things simple it is the same name as the schema class but without the word “Schema”. Note inside the round brackets we call the Schema class.
class ShowIpNbarDiscovery(ShowIpNbarDiscoverySchema):
"""Parser for show ip nbar protocol-discovery protocol on IOS-XE
parser class - implements detail parsing mechanisms for cli output.
"""
cli_command = 'show ip nbar protocol-discovery protocol'
You can also see we have our CLI command which is what you will enter into device.parse or your pyATS script to run the parser.
The next thing to look at is the function itself which runs the parser. At first glance this looks like a funny conditional (if statement) but my interpretation is this is a check to see if the command/parser has already been run because the default for this function is output=None. Assuming it’s the first time it’s been called then we execute the CLI command to get the raw CLI output.
Finally in this section we create a new (empty) dictionary object result_dict. I’ve seen it called a few other things (i.e take a look at other parsers for reference). However, this name sticks for me because it explains exactly what this is doing. At the end of the function we will be returning result_dict with a complete dictionary to the schema class which will validate the dictionary output matches the proposed schema.
def cli(self, output=None):
if output is None:
out = self.device.execute(self.cli_command)
else:
out = output
result_dict = {}
The next section maybe difficult depending on how familiar you are with using REGEX.
p1 = re.compile(r'^(?P<interface>Gig.+|Ten.+|Fast.+ |Port.+)')
p2 = re.compile(r'^(?P<protocol>[\w\-]+) +(?P<In_Packet_Count>[\d]+) +(?P<Out_Packet_Count>[\d]+)')
p3 = re.compile(r'^(?P<In_Byte_Count>[\d]+) +(?P<Out_Byte_Count>[\d]+)')
p4 = re.compile(r'^(?P<In_Bitrate>[\d]+) +(?P<Out_Bitrate>[\d]+)')
p5 = re.compile(r'^(?P<In_Bitrate_Max>[\d]+) +(?P<Out_Bitrate_Max>[\d]+)')We have to create variables using the “p<sequence order number>” standard e.g. p1,p2,p3 etc. We have re.compile to define a regular expression. What I can’t do is give you a massive overview on using REGEX, it’s too big a topic. That said, if you’re genuinely worried about REGEX then my advice would be to plagiarize others parsers for ideas rather than re-invent the wheel. I would also use regex101.com which is great at helping define your matching schema.
I will walk-through the ones I need for this command and how they are defined.
^ start of the line
(?P<name of the capture group><what I want to capture in REGEX>)
Variable p1
In this case the first line of interesting output is the interface hence I called my capture group interface. Next I am matching any interface names which start with the letters Gig or Fast or Port. The .+ in REGEX just means one or more characters. It’s safe to use this as that line only has the name of the interface and nothing else so it won’t over capture.
Variable p2
My first capture group is looking for the protocol name e.g. “ssh” which is a word.
\w matches any word character (equal to [a-zA-Z0-9_])
\- matches the hyphen character – literally (case sensitive)
I’ve included a hyphen in there because there might be a protocol name which has a hyphen in the name though I am unsure but it’s best to be safe here.
The remaining two capture groups In_Packet_Count and Out_Packet_count are on the same line as the protocol. The REGEX on those is very simple since they should always be a digit.
\d matches a digit (equal to [0-9])
+ special meaning to match one or more of the same
Variable p3,p4 and p5
All use \d to match digits for In Byte Count, Out Byte Count, In Bitrate, Out Bitrate, In Bitrate Max and Out Bitrate Max
These are more the same to match on. If you look at the show ip nbar protocol discovery output then these fields will make sense.
Writing your REGEX patterns may take sometime depending on your experience and comfortability of using them. I kept playing with regex101.com and trying adding and removing things when I was learning it. I now can read a REGEX pattern and know roughly what is going on without reading the helper text on regex101.com.
In Part 3, we will look at how these complied REGEX expressions are used to extract the CLI output into a dictionary using a standard framework.
Bite Size Automation 