Python for network engineers - telnetlib

The typical job of the network engineer is to manage the networks. To manage the network in traditional way, the network engineer connects to the CLI of the device and enters the series of commands. The most simple way to do the job is by using the telnet protocol. Python already includes telnet client in the standard library. The standard telnet library in python is called telnetlib. The goal of this post is to show, how to write a simple script, that connects to the router, executes some command and exits.

The telnetlib consists of python class Telnet. You have two options to initialise the Telnet instance and open the connection to the telnet client.

The first option is to create a instance of the Telnet and then use open() method to establish the connection.

conn = telnetlib.Telnet()
conn.open('192.168.35.121')

The other option is to establish connection during class initialisation.

conn = telnetlib.Telnet('192.168.35.121')

To read the data that is received from the connection, you can use read_until() method. You need to provide the expected string to the method. In this phase, we will assume that router is configured with AAA and "Username: " string is returned from the router. It is a good practice to include some timeout to the method, to prevent the blocking in the code. So our code will be:

conn.read_until('Username: ', 3)

The method will read until the given string is matched or until the timeout is reached. If there is no match the method will return whatever was received.

Now we are ready to send some data via telnet connection. You can use the write() method. The write() method expects the string that should be send to the telnet server. So in our example, we need to send username together with new line symbol (\n).

conn.write('cisco\n')

After you are finished it is advisable to close the connection using close() method.

conn.close()

So lets write a simple program that connects to the router, sends the username and password, close the connection.

import telnetlib

# Establish the connection
conn = telnetlib.Telnet('192.168.35.121')

# Read the output for the username request
output = conn.read_until('Username: ', 3)

# If 'Username: ' is found, send the username
if 'Username: ' in output:
    conn.write('cisco\n')

# Read the output for the password request
output = conn.read_until('Password: ', 3)

# If 'Password: ' is found, send the password
if 'Password: ' in output:
    conn.write('cisco\n')

# Read for hash prompt
output = conn.read_until('#', 3)

# Check for # in output
if '#' in output:
    conn.write('show ip interface brief\n')

# Read the output
output = conn.read_until('#', 3)

# Print the output
print output

# Close the connection
conn.close()

Sorting in python

How to sort lists in python? The easiest option is to use the sort method.

For example, to sort the list of vendors you can use the following procedure:

>>> devices = ['cisco', 'juniper', 'alcatel']
>>> devices.sort()
>>> devices
['alcatel', 'cisco', 'juniper']

The python sort the elements alphabetically. Ok that was a simple example. What if we have a list of IP addresses, like in example bellow.

>>> ipAddresses = ['1.1.1.1', '6.6.6.6', '10.10.10.10']
>>> ipAddresses.sort()
>>> ipAddresses
['1.1.1.1', '10.10.10.10', '6.6.6.6']

This is not correct. The problem is that python sorts elements in list alphabetically. This means that first letter in the string '10.10.10.10' is before the first letter in string '6.6.6.6'. How to solve this?

The solution is to use the key argument in the sort method. The key argument expects the function, which will be called for each element and the result of the function will be used to sort elements in the list.

We can transform the IP address into decimal form and then sort these values based on the decimal value. The solution for sorting the IP addresses can be:

>>> import ipaddr
>>> def transformToDecimal(ip):
...  ipAddress = ipaddr.IPv4Network(ip)
...  return int(ipAddress)
>>>
>>>
>>> ipAddresses
['1.1.1.1', '10.10.10.10', '6.6.6.6']
>>> ipAddresses.sort(key=transformToDecimal)
>>> ipAddresses
['1.1.1.1', '6.6.6.6', '10.10.10.10']

Now the result is as we have expected. The sort function was calling the transformToDecimal function for every element in the list. The function returned the decimal representation of the IP address, which was then used for sorting the elements in the list.

Now continue with a little more complex problem. How to sort the list of dictionaries by the values. For example, if we have the following list of dictionaries:

deviceList = [{'vendor': 'cisco', 'ipAddress': '1.1.1.1'}, {'vendor': 'juniper', 'ipAddress': '6.6.6.6'}, {'vendor': 'alcatel', 'ipAddress': '10.10.10.10'}]

How can we sort these dictionaries by the vendor. The solution could be to implement own function that return the vendor value. But the python has similiar function that is already built-in. The module is called operator. So the complete solution is:

>>> import operator
>>> getVendor = operator.itemgetter('vendor')
>>> deviceList
[{'ipAddress': '1.1.1.1', 'vendor': 'cisco'}, {'ipAddress': '6.6.6.6', 'vendor': 'juniper'}, {'ipAddress': '10.10.10.10', 'vendor': 'alcatel'}]
>>> deviceList.sort(key=getVendor)
>>> deviceList
[{'ipAddress': '10.10.10.10', 'vendor': 'alcatel'}, {'ipAddress': '1.1.1.1', 'vendor': 'cisco'}, {'ipAddress': '6.6.6.6', 'vendor': 'juniper'}]

The itemgetter method create a function that return dictionary value, of the key that is specified as an argument. In our example, we are using 'vendor' key.

So the sort option is very simple for the simple lists. But for the more complex lists you can specify your own sorting key, which could be very useful.

Using exaBGP to control your routers

ExaBGP is a python implementation of the BGP protocol. You can run exaBGP on the server and establish BGP sessions to your routers. You can find the exaBGP source on the github.

https://github.com/Exa-Networks/exabgp

What you can do with it? You can for example use it to monitor the state of your network or to manipulate routes. You can find many use cases on the following link:

https://github.com/Exa-Networks/exabgp/wiki/Related-articles

To install exaBGP on your server use python pip.

pip install exabgp

The advantage of exaBGP is that it is very programmable. It transform BGP messages to text or JSON files. You can communicate with exaBGP with the API. To send commands to the exaBGP write these commands to STDOUT. To receive the data read the STDIN.

To begin with the exaBGP you need a configuration file. The configuration file specify the basic BGP settings for the server and basic neighbor settings. The example of the file is:

neighbor 192.168.1.1 {
    router-id 192.168.1.2;
    local-address 192.168.1.2;
    local-as 1;
    peer-as 1;
    graceful-restart;

    process routes {
        run ./routes.py
    }
}

With the neighbor command you specify the IP address of the neighbor. Inside the neighbor configuration you specify the router-id of the server, the IP address, which will be used by the server, local and remote AS number (you can use iBGP or eBGP session). The process command is mandatory. With process command you specify which external script will exaBGP run. You can run any script you want. You can use this script to write to the STDOUT, which will be captured by exaBGP and use this command to execute some tasks. You can run multiple scripts by using multiple process commands.

The example of the routes.py is:

import sys
import time

routes = [
'announce route 1.1.1.1/32 next-hop 192.168.1.10',
]

time.sleep(1)

for route in routes:
    sys.stdout.write(route + '\n')
    sys.stdout.flush()
    time.sleep(1)

while True:
    time.sleep(1)

The exaBGP will execute this script and this script will write the ‘announce …’ command to the STDOUT. This command will tell the exaBGP to announce this route to the BGP peer. And if you check your router after announcing this route, you can see this route in the BGP table.

You can also construct your configuration with the group parameters:

group DEMO {
    router-id 192.168.1.2;
    local-address 192.168.1.2;
    graceful-restart;

    neighbor 192.168.1.10 {
        local-as 65010;
        peer-as 65010;
    }

    neighbor 192.168.1.11 {
        local-as 65011;
        peer-as 65011;
    }

    process routes {
         run ./routes.py;
    }
}

As you can see, you can use different AS numbers on the BGP server. That is really the advantage of using exaBGP.

To run an exaBGP server use the following command:

exabgp demo.conf

This command will run the server and establish the BGP sessions with your routers.

ExaBGP is a great tool, if you want to have the programmable BGP implementation. The only thing that I am missing is the lack of documentation, and examples.

Use ansible to update host file on the servers

I had some time to play around with Ansible. Ansible is a great tool, which allows you to automate your environment.

I came to a problem, where I need to distribute host file to all machines in inventory and update this host file with host specific variables, which are located in inventory as well. Here is the guide that shows you how to create host file and distribute this file to all hosts that are located in the inventory.

To achieve the goal, I have created the ansible host file, with all hosts in my network. I have grouped the the servers based on server roles. It looks like this:

$cat ansible_hosts
[controller]
controller

[network]
network

[compute]
compute1
compute2

I have also created the subfolder, which is located in the same folder as ansible_hosts file. I have named this folder host_vars. Ansible automatically load files located in that folder and associate variables with the host in inventory file. The name of the file must be the same as name of the host in inventory. I put the following variables in the file:

$cat host_vars/controller
---
hostname: controller
ip_address: 10.0.0.101

You can use ansible template module, which takes file and update file with variables. Ansible uses Jinja2 templating system. I took the default ubuntu hosts file and updated file with the following lines:

127.0.0.1 localhost
127.0.1.1 {{ hostname }}

{% for host in groups['all'] %}
{{ hostvars[host]['ip_address']  }} {{ hostvars[host]['hostname']  }}
{% endfor %}

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

The Jinja2 templating system replace {{ hostname }} with the variable hostname of the current host. Another interesting block is for loop. The for loop goes through all hosts that are listed in ansible inventory file. You can also replace the 'all' with the specific group, which will be used to go only through hosts listed in that group. The hostvars is reserved dictionary, which holds all variables for all hosts. You can access specific variable with correct keys.

Now I have created the playbook which will use this template.

---
- hosts: all
  remote_user: ansible
  tasks:

   # Create hosts file from variables and copy file to remote destination /etc/hosts
   - template: src=./templates/hosts dest=/etc/hosts
     become: yes
     become_method: sudo

You should run this playbook with the command:

$ ansible-playbook openstack_install.yml
...
TASK [template dest=/etc/hosts src=./templates/hosts] **************************
changed: [compute1]
changed: [compute2]
changed: [controller]
changed: [network]
...

If you display the host file on one of the server, the hosts file is updated correctly.

ansible@controller:~$ cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 controller

10.0.0.131 compute1
10.0.0.132 compute2
10.0.0.101 controller
10.0.0.111 network

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

This is the example of the usage of the Jinja2 templating system with ansible. The templating system is very powerful which allows you to create complex template and updated these templates with correct variables.

Automatic connectivity test on the router with EEM

Did you ever want to perform constant connectivity test to the different hosts on the Internet? You can use Embedded Event Manager for the task. EEM is part of the IOS software. I will show you how to use the EEM and TCL scripting for such task.

First you need to create TCL script, that will be used for connectivity test. You can find it bellow.

Router#more ping.tcl
set hosts {
{4.2.2.1}
{8.8.8.8}
{8.8.4.4}
}
set counter 0
set sucess_counter 0
set failed_hosts ""

foreach ip $hosts {
incr counter 1
set result [exec ping $ip]
puts $result
if {[regexp {!!!} $result]} {
incr sucess_counter 1
} else {
puts $ip
set failed_hosts "$failed_hosts  $ip"
}
}

set msg_output "Success rate $sucess_counter/$counter. Failed hosts: $failed_hosts"
[ios_config "event manager environment syslog_msg_output $msg_output"]

Let me first explain the flow of the script. The hosts variable is set with the command set host {}. You can include as many hosts on the Internet as you want. After that the three other variables are initialized. In the main part of the script ping command is performed for every hosts. If the result is at least three !, the test is successful and success_counter is increased. Otherwise failed host is added to the variable failed_hosts. After for loop is performed the text for the syslog message is created. The last step of the script is to add event manager environment variable to running config of the router. The variable name is syslog_msg_output and the value is the string msg_output.

After script is created and saved to the flash of the router, you need to tell your router what to do with the script. You can enter the following commands to the router running config.

event manager applet PING_TEST
 event timer watchdog time 300 maxrun 250
 action 0.5 cli command "enable"
 action 1.0 cli command "tclsh flash:ping.tcl"
 action 2.0 syslog msg "$syslog_msg_output$"
 action 3.0 cli command "configure terminal"
 action 4.0 cli command "no event manager environment syslog_msg_output"
 action 5.0 cli command "end"
 action 6.0 cli command "exit"


Let me explain what the commands above means. First create event manager applet. I have created the applet with the name PING_TEST. Next specify when this applet will be run. I run this applet every 300 seconds and let it run for the max of 250 seconds. It is important to trim these timers since you want to script to end before timeout (maxrun) expires. The default value is 20 seconds. After event timer has been set you can specify commands to be run in the applet. To run a script use "tclsh flash:ping.tcl" command and to create syslog message use syslog msg "$syslog_msg_output$". This will create syslog message with the text from event manager variable syslog_msg_output. After syslog message is created you can delete variable end exit.

It is time to test if script is working as expected. You can check logging buffer and you can see messages like this:

 %HA_EM-6-LOG: PING_TEST: Success rate 3/3. Failed hosts: $

If any of the hosts fail it will be listed under failed hosts and success rate will be adjusted accordingly to the test.

You are free to use this script in your environment. I hope you will find it usefully.

Automatic Anyconnect VPN connection on untrusted networks

Often it is needed for the remote workers to have automatic VPN connection when they are outside of the company. You can use ASA and Anyconnect client to deploy such solution. In this blog post I will show you have to configure Cisco ASA to support Anyconnect for such deployment. Certificates will be used for authentication.

The first thing is to configure SSL VPN server on the Cisco ASA to use certificates for the authentication. I will skip certificate issuing procedure. Bellow you will find basic configuration for SSL VPN on the ASA.

webvpn
 enable outside
 anyconnect image disk0:/anyconnect-win-3.1.04072-k9.pkg 1
 anyconnect enable

ssl trust-point SSLVPN_CERT outside

group-policy SSLVPN_GP attributes
 dns-server value 192.168.1.10 192.168.1.11
 vpn-filter value SSLVPN_FW
 vpn-tunnel-protocol ssl-client
 split-tunnel-policy tunnelspecified
 split-tunnel-network-list value SSLVPN_SPLIT
 default-domain value example.com
 address-pools value SSLVPN_POOL

tunnel-group SSLVPN_TG type remote-access
tunnel-group SSLVPN_TG general-attributes
 default-group-policy SSLVPN_GP
tunnel-group SSLVPN_TG webvpn-attributes
 authentication certificate
 group-url https://vpn.example.com/auto enable

This is basic SSLVPN configuration and you can try to connect on the outside interface. The next step is to configure Anyconnect profile which will create policy for automatic VPN connection on untrusted networks. You can create Anyconnect profile via ASDM.

When you are connected to ASA with ASDM, click Configuration -> Remote Access VPN -> Network(Client) Access -> AnyConnect Client Profile. In this configuration mode you can add new Anyconnect profile. Click Add button and choose Profile Name and Profile Location. You can also apply this profile to Group Policy you have created in the previous step. But this could be also added later with the command. Click OK and Apply.

group-policy SSLVPN_GP attributes
 webvpn
  anyconnect profiles value AUTO type user

 

Now double click on the profile that has been created and configure profile.

Preference (Part 1)

• Select User as certificate store.

 

Preference (Part 2)

• Uncheck Disable Automatic Certificate Selection which will configure Anyconnect to automatically select correct certificate.
• Check Automatic VPN Policy and select Disconnect on Trusted Network Policy and Connect on Untrusted Network Policy. You must also enter DNS domain name for your trusted network and you should also add DNS servers.

 

Certificate Matching

• In this tab you can configure which certificate to use when connecting to the SSL VPN server. I have selected the ISSUER-CN.

Server List

• You must add at least one server otherwise Certificate Matching will be ignored. Configure the same display name and host address as used in the tunnel-group.

 

After all this has been configured you are ready to test your connection. First you need to connect with Anyconnect manually, so that Anyconnect client download profile. After that you can connect to the Untrusted network and test if your Anyconnect client will connect automatically.