Event-driven network automation using Slack, hubot, StackStorm and Ansible

April 1, 2016

Event-driven automation is a very interesting topic. Think about executing task X when a POST-request gets fired to an application. Or about executing task Y as soon as a special mail gets obtained. I thought about bringing this topic together with network-automation and the things I have learned about Ansible and Co. I also got inspired by some great guys who are active in the networktocode slack-team. For example there is already a blog post published by @dbarrosop which covers a similar topic. If you are interested in network-automation networktocode is the place to go. StackStorm is a tool for event-driven automation. The tool is very successful and has a big company behind. A few days before the start-up just got bought by Brocade, which let me hope about getting more networking-content within StackStorm in the future. The community-edition will still be available as open-source product at no charge.

I started a project in my development environment with the goal to automate Cisco IOS-devices over simple chat-messages in Slack. At the moment of writing this post I had build actions and aliases in StackStorm to add or delete vlans and to ensure a base config on my network switches. In this blog post I want to show you the possibilities and how you can get this working in your own environment. The following picture illustrates the automation workflow.

workflow

Summarized we have to install and configure the following components:

  • Slack team
  • Slack hubot plugin
  • An Ubuntu 14.04. server with the help of vagrant
  • StackStorm
  • Custom StackStorm packs
  • Ansible

Lets get started with the Slack components

Go to slack.com and create a new team. In my environment the team is named itsnetwork. After creating the team go to team settings->Configure Apps, search for hubot and add it to your team. Also copy the API Token to a temporary text file. Hubot is an open source bot, written in CoffeeScript on Node.js and a standardized way to share scripts between everyones robots. We will use it as a gateway between Slack and StackStorm.

slack-hubot

Now invite your hubot to one of your team-channels (replace the bot-name with your own):

/invite @simons-hubot

Set up Ubuntu 14.04. with vagrant

We need a server where StackStorm will run. In our testing environment vagrant will be used to setup an Ubuntu 14.04. server. Follow the instructions on the Vagrant-Site to install Vagrant on your operating system. Also install a vagrant provider, which is mostly virtualbox. I recommend you to follow the install-instructions on the links and not to install the software with package-managers of your operating system.

After installing Vagrant and virtualbox now create a new folder called st2 and create the Vagrantfile. Edit the public_network ip to an ip which is reachable in your local LAN. This will be the ip-address of your server. Then build the VM.

$ mkdir st2
$ cd st2
$ echo "VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
 
   config.vm.box = "ubuntu/trusty64"
   config.vm.box_download_insecure = true
   config.vm.network "public_network", ip: "192.168.0.21"
 
   config.vm.provider "virtualbox" do |vb|
           vb.memory = "2404"
   end
end" >> Vagrantfile
$ vagrant up

Install StackStorm community edition

SSH to your vagrant VM with vagrant ssh and follow the official install instructions for StackStorm on Ubuntu. If you have trouble installing or starting mistral please check your installed locales. In the Setup ChatOps section you will also install hubot and the StackStorm-hubot-plugin. These things are included in the st2chatops package which gets installed with apt.

Now also update the chatops pack files to the most recent version, because the one installed via apt-get includes a bug (replaces \n with @).

After setting up all those things please play around with some st2 commands on the StackStorm-VM or look at the web-interface.

stackstorm-webui

Install the custom StackStorm packs

I have built two custom StackStorm packs which we will install now. Packs could contain StackStorm actions, aliases, rules and sensors. For our purpose the ansible playbooks and roles will be included additionaly. The packs could be found on github:

Install them on your VM with:

$ st2 run packs.install packs=st2-netops-vlan repo_url=https://github.com/smnmtzgr/st2-netops-vlan
$ st2 run packs.install packs=st2-netops-base_config repo_url=https://github.com/smnmtzgr/st2-netops-base_config

Edit the following things:

  • insert your hosts to the ansible-host file located at /opt/stackstorm/packs/st2-netops-base_config/playbooks/hosts
  • insert your hosts to the ansible-host file located at /opt/stackstorm/packs/st2-netops-vlan/playbooks/hosts
  • change user and password in the ansible group_vars file located at /opt/stackstorm/packs/st2-netops-base_config/playbooks/group_vars/ios.yaml
  • change user and password in the ansible group_vars file located at /opt/stackstorm/packs/st2-netops-vlan/playbooks/group_vars/ios.yaml

Now we have to setup Ansible on the StackStorm VM. The devel-branch includes all needed network-modules:

$ git clone http://github.com/ansible/ansible.git
$ cd ansible
$ git checkout devel
$ git submodule init
$ git submodule update
$ pip uninstall ansible
$ python setup.py install
$ ansible --version
ansible 2.0.1.0 (stable-2.0-network 49f7e6a12c) last updated 2016/03/01 20:57:24 (GMT +200)
  lib/ansible/modules/core: (detached HEAD 72540d1f0c) last updated 2016/03/01 20:57:27 (GMT +200)
  lib/ansible/modules/extras: (detached HEAD 51cddf2b35) last updated 2016/03/01 20:57:28 (GMT +200)

Setup Slack-integration within the StackStorm-hubot-plugin

If not already done while following the StackStorm install instructions now edit the /opt/stackstorm/chatops/st2chatops.env file, comment out the Slack-lines and insert your hubot API Token. Also insert the password for your st2admin-user. My file looks like this (API Token is changed):

#####################################################################
# Hubot settings

# set if you don’t have a valid SSL certificate.
export NODE_TLS_REJECT_UNAUTHORIZED=0

# Hubot port - must be accessible from StackStorm
export EXPRESS_PORT=8081

# Log level
export HUBOT_LOG_LEVEL=debug

# Bot name
export HUBOT_NAME=stanley
export HUBOT_ALIAS=!

######################################################################
# StackStorm settings

# StackStorm api endpoint. (Don’t use `localhost` as it would point to the Docker container).
export ST2_API=https://${ST2_HOSTNAME}/api

# StackStorm auth endpoint. (Don’t use `localhost` as it would point to the Docker container).
export ST2_AUTH_URL=https://${ST2_HOSTNAME}/auth

# ST2 credentials
export ST2_AUTH_USERNAME=st2admin
export ST2_AUTH_PASSWORD=<INSERT YOUR PASSWORD>

# Public URL of StackStorm instance: used it to offer links to execution details in a chat.
#export ST2_WEBUI_URL=https://${ST2_HOSTNAME}
export ST2_WEBUI_URL=https://192.168.0.21

######################################################################
# Chat service adapter settings

# Uncomment one of the adapter blocks below.
# Currently supported: slack, hipchat, xmpp, yammer, irc, flowdock.

# Slack settings (https://github.com/slackhq/hubot-slack):
#
export HUBOT_ADAPTER=slack
export HUBOT_SLACK_TOKEN=<INSERT YOUR API TOKEN HERE>

Now restart st2chatops with sudo service st2chatops restart.

Test it

Go back to your Slack-team and execute some commands. After playing around check on your network-devices if the base_config is ensured and if the vlans you added/deleted via chat-commands are really added/deleted to/from your devices. Here are some screenshots and outputs of my results:

hubot-help hubot-addvlan

switch-01#sh vlan brief

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Fa0/1, Fa0/2, Fa0/3, Fa0/4, Fa0/5, Fa0/6, Fa0/7, Fa0/8, Gi0/1
5    test                             active    
10   LX                               active    
22   blogpost                         active    
700  fw                               active    
1002 fddi-default                     act/unsup 
1003 token-ring-default               act/unsup 
1004 fddinet-default                  act/unsup 
1005 trnet-default                    act/unsup 

hubot-deletevlan

switch-01#sh vlan brief

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Fa0/1, Fa0/2, Fa0/3, Fa0/4, Fa0/5, Fa0/6, Fa0/7, Fa0/8, Gi0/1
5    test                             active    
10   LX                               active    
700  fw                               active    
1002 fddi-default                     act/unsup 
1003 token-ring-default               act/unsup 
1004 fddinet-default                  act/unsup 
1005 trnet-default                    act/unsup 

hubot-ensureconfig

switch-01#sh run | i snmp
snmp-server community public RO
snmp-server community private RW
snmp-server community operations RW
snmp-server location data center
snmp-server contact network operations
switch-01#sh run | i ntp 
ntp source Vlan1
ntp server 1.1.1.1
ntp server 2.2.2.2
ntp server 3.3.3.3
ntp server 4.4.4.4

Summary

This use-case is only a very small and easy one but shows a lot of the power and possibilities you have with todays tools. StackStorm is easily said an orchestration tool which can control everything what happens on the basis of events. To clear things up: in our case we simply used some chat-commands as manual API-calls. Real events are normally not manually created by a human being, they are generated by another machine. For example in a real-world scenario it would be possible that a monitoring tool like icinga - in the case of a special event (host down, interface down…) - generates a HTTP-POST request to StackStorm which will then trigger some actions. Your fantasy should not have any constraints! :)

Miscellaneous

Good sources I used to get started with those things are:

Back...