Parsing JUNOS rpc-reply with BeautifulSoup4

What I like about JUNOS is that it’s designed with programmability in mind. All the show commands have xml output that can be requested using RPC. They also have rich and well-documented APIs and python library(PyEZ). Possibilities are endless once we have the XML output. You can write apps and integrate the Junos API within.

BeatifulSoup4 is a popular library used to parse data out of XML and HTML markup. We’ll use BS4 to extract information out of JUNOS RPC-Reply which encoded in XML. I have following RPC string which displays the output of “show system core-dumps”. Basically, I want to get the list of core-dump files and time they were generated.

<rpc-reply xmlns:junos="http://xml.juniper.net/junos/12.1X46/junos">
    <directory-list root-path="/var/crash/*core*" junos:seconds="1549057170" junos:style="verbose">
        <output>
            /var/crash/*core*: No such file or directory
        </output>
        <directory name="">
            <file-information>
                <file-name>/var/tmp/rpd.core.0.gz</file-name>
                <file-permissions junos:format="-rw-rw----">660</file-permissions>
                <file-owner>root</file-owner>
               <file-group>wheel</file-group>
                <file-links>1</file-links>
                <file-size>3556567</file-size>
                <file-date junos:format="Feb 1  21:37">1549057078</file-date>
            </file-information>
            <output>
                /var/tmp/pics/*core*: No such file or directory
            </output>
            <output>
                /var/crash/kernel.*: No such file or directory
            </output>
            <output>
                /tftpboot/corefiles/*core*: No such file or directory
            </output>
            <total-files>1</total-files>
        </directory>
    </directory-list>
    <cli>
        <banner></banner>
    </cli>
</rpc-reply>

We’ll import the bs4 module. For the purpose of this post, XML Reply is stored as a string but you can request the output using JUNOS PYEZ module. 

>>> from bs4 import BeautifulSoup as Soup
>>> rpcreply = '<rpc-reply xmlns:junos="http://xml.juniper.net/junos/12.1X46/junos"><directory-list root-path="/var/crash/*core*" seconds="1549057102" style="verbose"><output>/var/crash/*core*: No such file or directory</output><directory name=""><file-information><file-name>/var/tmp/rpd.core.0.gz</file-name><file-permissions format="-rw-rw----">660</file-permissions><file-owner>root</file-owner><file-group>wheel</file-group><file-links>1</file-links><file-size>3556567</file-size><file-date format="Feb 1  21:37">1549057078</file-date></file-information><output>/var/tmp/pics/*core*: No such file or directory</output><output>/var/crash/kernel.*: No such file or directory</output><output>/tftpboot/corefiles/*core*: No such file or directory</output><total-files>1</total-files></directory></directory-list>'

Initialize the Soup class. 

>>> soup = Soup(rpcreply,'xml’)
>>> soup
<?xml version="1.0" encoding="utf-8"?>
<rpc-reply xmlns:junos="http://xml.juniper.net/junos/12.1X46/junos"><directory-list root-path="/var/crash/*core*" seconds="1549057102" style="verbose"><output>/var/crash/*core*: No such file or directory</output><directory name=""><file-information><file-name>/var/tmp/rpd.core.0.gz</file-name><file-permissions format="-rw-rw----">660</file-permissions><file-owner>root</file-owner><file-group>wheel</file-group><file-links>1</file-links><file-size>3556567</file-size><file-date format="Feb 1  21:37">1549057078</file-date></file-information><output>/var/tmp/pics/*core*: No such file or directory</output><output>/var/crash/kernel.*: No such file or directory</output><output>/tftpboot/corefiles/*core*: No such file or directory</output><total-files>1</total-files></directory></directory-list></rpc-reply>

select() method returns a list containing BeautifulSoup objects which identifies multiple occurrence of the specified tag, within the XML hierarchy. Let’s extract the content of file-information tag, which contains core-dump file-name and the time the it was generated.

>>> soup.select("file-information")
[<file-information><file-name>/var/tmp/rpd.core.0.gz</file-name><file-permissions format="-rw-rw----">660</file-permissions><file-owner>root</file-owner><file-group>wheel</file-group><file-links>1</file-links><file-size>3556567</file-size><file-date format="Feb 1  21:37">1549057078</file-date></file-information>]
>>> soup.select("file-information")[0]
<file-information><file-name>/var/tmp/rpd.core.0.gz</file-name><file-permissions format="-rw-rw----">660</file-permissions><file-owner>root</file-owner><file-group>wheel</file-group><file-links>1</file-links><file-size>3556567</file-size><file-date format="Feb 1  21:37">1549057078</file-date></file-information>

As you can see, the select() method has returned a list with a single BeautifulSoup element, since our reply contained output with a single file. To further extract the file-name and file-information; we’ll call the select method of the element.

>>> soup.select("file-information")[0].select("file-name")
[<file-name>/var/tmp/rpd.core.0.gz</file-name>]
>>> soup.select("file-information")[0].select("file-date")
[<file-date format="Feb 1  21:37">1549057078</file-date>]

To extract the text payload of the tags we’ll use the get_text()method of the elements.

>>> soup.select("file-information")[0].select("file-name")[0].get_text()
'/var/tmp/rpd.core.0.gz’
>>> soup.select("file-information")[0].select("file-date")[0].get_text()
'1549057078'

That gave us the file name, which identifies the core-dump file name and path, and file-date, which contains the time the core-dump file was generated in EPOCH format.

This was just a POC but you can do anything with this information. Encode it into a JSON and throw it at your javascript frontend or store it into a database.. possibilities are endless!

JUNOS XML RPC Reply can be parsed using BeautifulSoup to extract specific information out of the reply. This is extremely useful, especially if you want to deploy your own application which interfaces with the network.

 

Mihir

 

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.