In one of my previous articles, I elaborated on a setup to create DC fabric simulations with GNS3 and OpenSwitch. I also promised to follow up with some post about using Ansible with it.
Well, I have been a bit distracted with some changes to the setup before moving into the Ansible details (to say true, I got the Ansible article almost ready, but things are moving so fast, that I keep rewriting it).
One of the things that ‘change everything’ recently was the release of GNS3 1.5. This is a great release that includes several features that makes using OpenSwitch with GNS3 awesome:
Let’s see how to use this new release with OpenSwitch Appliance Container.
The first recommendation when using the container setup for GNS3, is to use the GNS3 VM (even when using a native Linux host). Why?
Since we will be using qemu appliances inside the GNS3 VM, you need a virtualization platform that supports nested virtualization. That means you are either using a VMware desktop solution (Workstation, Player or Fusion), or if using ESXi, be sure to enable the VHV flag.
Download the VM from here. Be sure to download the right image for either desktop solution or ESXi.
You will need at least version 1.5.0, but if you are behind a corporate firewall, you will need version 1.5.1 that fixes a bug when configuring the proxy settings.
After downloading the GNS3 VM appliance, import it into your virtualization solution, and reconfigure with this minimal hardware requirements:
If you are behind a corporate firewall, be sure to turn on the VM, and using the menu displayed when the VM boots, configure the proxy settings.
If you haven’t already, install GNS3 downloading from here (be sure to use the same version as the VM image that you downloaded).
When configuring GNS3, the wizard doesn’t allow you to configure a remote server GNS3 VM, but you can abort the wizard, and add it later from the preferences panel (the IP address of the VM is show on the information page displayed by the VM on boot, or available from the text menu on the VM as well).
In order to use OpenSwitch as a container with GNS3, you have to main options: use an existing image (easier one), or build a custom one from scratch (useful for developers).
Let’s focus on the simple procedure first. Fortunately we no longer need to specify multiple parameters on the UI to create a template, since for Docker containers we can use a pre-defined template appliance (.gns3a file), that will do all the configuration for you.
First, download a GNS3 appliance template from the list available here.
Next, import it into GNS3: there is a new ‘Import appliance template’ button under the device panel.
Select ‘Import an appliance template file’
Select the template that you downloaded, and follow the rest of the wizard. Select to run on the remote server running the GNS3 VM.
Once you are done with the import, you can download a custom stencil for OpenSwitch Docker Appliance from here, and customize the template to use it.
The template uses by default 8 ports, but that can be adjusted in per machine instance.
That’s all it takes, you are ready to drag & drop container instances into GNS3!
In the previous step where we used the .gns3a file, it points to a published version of the container in the Docker Hub, but you may want to create your own container from scratch with the latest code or modifications. If that is not the case, you can skip this section.
First you will have to build an image for the appliance target as explained in previous articles (you can also download the tarball for a periodic build from the artifact archives).
Now, the default tarball created for the appliance is designed to run in a VM, so it needs some small modifications to run properly on a container. I have automated the steps in a docker file here.
Here are the example commands to create a custom image with the latest code from
master and push into the docker hub (in this example I use my docker hub account):
1
2
3
4
5
6
mkdir ops-gns3-docker
curl https://archive.openswitch.net/artifacts/periodic/master/latest/appliance/openswitch-appliance-image-appliance-ops-0.4.0-master+2016063012.tar.gz > openswitch-appliance-image.tar.gz
wget https://raw.githubusercontent.com/ddompe/ops-gns3-docker/master/openswitch/Dockerfile
docker build -t ddompe/openswitch-appliance:latest .
docker login
docker push ddompe/openswitch-appliance:latest
Now you can modify the gns3a file provided in the previous step and point to your customized image to import it into GNS3.
Now, let’s create a topology for creating a DC Fabric simulation with Ansible similar to the one described in the previous article using virtual machine appliances.
I have created a docker container that already automated all the steps detailed in the previous article, and is suitable to be run as a systemd container. The Dockerfile could be found here, and the GNS3 template file here.
Unlike the previous examples using VirtualBox, we cannot longer use the cloud element to get a NAT connection to the external world, so instead we will use the GNS3 Internet VM Appliance.
After installing those two templates, we can recreate the DC fabric similar to the one from the previous article, with the difference that is (almost) all containers.
Now comes another cool part, you can export and import the project to share with others.
So, you can export the project by using ‘Export portable project’ on the file menu. It will ask you if you wan to include any base image, which for our example I would recommend we do (this will pack the qemu image used by the GNS3 Internet Appliance). Docker images won’t get included on the file, but instead will be automatically downloaded from the docker hub when someone imports the project.
Talking about importing the project back, there is currently a bug when importing the project and downloading the docker images, where certain imports may timeout. This is fixed for 1.5.1, but in the meantime you can workaround by deleting the project that failed to import and re-import it again (since the docker images should be already downloaded, it won’t fail the second time).
The portable project file for the example on this article is here, in case you want to get up an running.
Isn’t all of this really exciting? Thanks to the GNS3 team for their awesome work and their responsiveness to fix the bug/enhancements that I keep throwing at them.
In the next article I will continue with explaining how to use Ansible to deploy configuration to the topology we build here.
Stay tuned.
]]>Note: This article was originally published here.
Before moving on the next post to continue our saga of OpenSwitch Simulations with GNS3, I wanted to take a quick deviation to document a subject that gets a lot of attention these days: P4.
In case you have been missing all the action around P4, the 30,000 feet view is that it’s a language to describe forwarding pipelines (and no, it’s not the same as OpenFlow, that is useful for programming entries in almost-always-pre-defined pipelines). One of the (many) nice things about this is that you can potentially ‘compile’ your pipeline definition into an executable program that provides a functional simulation of a P4-based ASIC. Did I mention the tools for doing all of this are available as open source?
Well, as I mention in previous posts, OpenSwitch provides a simulation environment that provides a software data path for testing and training purposes, and we package it as either container or VM appliance. So far the default software data path is provided by running an stock OpenVSwitch that’s under control of the switchd plugin for simulation.
However, since Barefoot Networks is a contributing member to the OpenSwitch project, they have been adding support for using a P4 backend that allows replace the simulation ‘ASIC’ to a P4 functional model of an ASIC. Some details about it can be found in their presentation from the last OpenSwitch Meetup (if you are interested on OpenSwitch, there were several interesting presentations on that Meetup. You can watch them here).
Since OpenSwitch is not married to a particular interface for the ASIC SDK (reports otherwise are greatly exaggerated!), the P4 backend provides a native plugin for switchd to use their ‘PD’ APIs directly.
OK, OK, how to run with the P4 data path? OpenSwitch still doesn’t provide P4 periodic images, so you have to build the code yourself. These are the steps you need to do to flip into the P4 backend (assuming you already have a working build environment for either container or appliance):
1 2 |
|
Now, run ‘make’ and go for a coffee while the image get’s build. It wasn’t that hard, was it?
One more thing™, if you are using the appliance environment, verify your code already includes this commit, or otherwise cherry-pick the change.
Pretty much the same way that you use the regular simulator image. If you are using the P4 Image with GNS3, you can download this special edition stencil I did for it.
One interesting fact that you may notice when using the P4 simulator, is that control packets into the CPU have higher latency. This is caused because the PDUs are going thru a TAP device from the switch process (in userspace) into the netdev interfaces in the SW namespace.
This kind of experimentation wasn’t possible a couple years ago, and software stacks like SAI and P4 are changing the landscape along with OpenSwitch. Think that you can change the forwarding data path implementation, rebuild the simulator, and integrate back again with your NOS stack! is mind boggling.
So, indeed, ‘software is eating the world’. Happy hacking!
]]>Note: This article was originally published here.
In the previous post I covered the basics about setting up the OpenSwitch Appliance using GNS3. The setup was fairly simple: two switches connected to each other and exchanging LLDP packets. In this post we will setup a more elaborate network to simulate a DC fabric (although it may be a bit overkill of a setup). The setup will be the basis for the next posts about configuring this fabric using Ansible.
One of the first questions when setting up a complex topology with GNS3 that most people will do is: how do I connect it to the external world outside of the simulation? For VirtualBox machines that we are using, the options are limited. The one I found to work reliably across platforms was to use a NAT connection. This has the disadvantage that we have limited connectivity from the external world toward the internal network, but this could be also a security advantage to prevent accidental propagation of control protocols from our simulated environment.
Since the purpose of this lab is going to be to play with Ansible, we are going to need a Linux machine to run it. So, we will setup the following network:
Let’s elaborate on the setup details:
So, let’s get started with the setup!
You should start by dragging a cloud element and configuring it to have a network interface of type NAT:
Now we need to create the template for the Ubuntu VM that we will connect to it.
As I mention, we will be using an Ubuntu 16.04 (state of the art!) Server VM. Don’t worry, you won’t have to install it. I have pre-package it into a nice OVA (sorry tested only with VirtualBox, you may need to do some work if want to use VMware). Download it from here.
After downloading the OVA file, import it into VirtualBox, and then proceed to create a template in GNS3 for the machine like we did for OpenSwitch in the previous post. Here is the summary of the steps and configuration:
Now you can drag an instance of it into your project and setup the connections as detailed previously. This image is configured with serial port console enabled, so after starting the VM, you can open the console from GNS3 and get access. The default user/password is ubuntu/ubuntu.
Now that you have access to the console, let’s do some initial setup. First you may want to tell the serial terminal the size so that the output is not messed up. In my case I just sized my console window to 120 columns and 35 lines (I use the OS X Terminal app, and the tittle usually says the dimensions):
1
|
|
Next we want to install the latest software updates for this image:
1 2 |
|
Next, we want to configure the second network interface facing towards SW1. You will have to run the ifconfig command to find the name of the second interface:
1
|
|
On this case my second interface is called enp0s8. I will then configure the interface to an static IP address and bring the interface up:
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Next we want to install the DHPC server and DNS relay (DNSmasq)
1
|
|
Then proceed to configure the dhcp server modifying the file /etc/dhcp/dhcpd.conf with your favorite editor (if you are not a vim expert, the nano text editor could be easier). This are the contents you will need:
1 2 3 4 5 6 |
|
Now you can start the server:
1
|
|
If you want the OpenSwitch boxes to have access to the external network over the management port, you may want to enable IP masquerading on the box. Adjust the following script to the name of your network interfaces:
1 2 3 4 5 6 |
|
Now that we have the Cloud element and the Ubuntu VM, you need to drag the GNS3 ethernet switch element to connect the second interface from the Ubuntu VM with it, along with all the management ports of the OpenSwitch instances. You can now proceed to power up the OpenSwitch instances and login into their consoles (user/password is netop), and verify the management interface got an IP address from our control VM:
1 2 3 4 5 6 7 8 9 10 |
|
You may also test login with user root (no password) into a bash shell and the networking of the management domain should be able to reach the external network.
Now that we have a management network up an running, we want to configure the fabric to move packets around. At this point most networking engineers will log into the CLI and start configuring, but we went thru all this setup of a management network to do something more interesting…
In our next post we will do this instead using Ansible, which is a pretty good tool for managing cattle.
]]>Note: This article was originally published here.
Update:This post has been updated to account for some recent changes in the appliance configuration (support for up to 7 front ports). In my previous post I described my developer setup to work with OpenSwitch. At the end of my post I showed how to download the build system, and configure and build an ‘appliance’ image.
The appliance is a virtual machine image (in OVA format) that could be run on VirtualBox or VMware (on this articule I will focus on VirtualBox) and provides a software datapath (based in OVS right now, but P4 support it’s landing soon). All the rest of the OpenSwitch stack is the same that you will see in a real hardware, and obviously the software datapath has certain limitations and features not implemented.
Despite his limitations, the appliance is a really nice way to get your hands into OpenSwitch without having real hardware.
If you are using the development environment, you can find the appliance .ova file on the images directory after completing the build, but otherwise you can also download a periodic image from the project archives (keep in mind this is a developer snapshot, so things may be broken or uncomplete sometimes).
The Appliance has currently 4 8
network ports (this is the max number of interfaces supported by VirtualBox): eth0 to eth7. The
port ‘eth0’ will be the management port, and the other ones are ‘front
ports’.
The first good use for the appliance is that you can import it into VirtualBox, and by using ‘bridged’ interface with a dedicated NIC (for example an USB ethernet dongle), you can make the appliance talk with a real piece of hardware, like a traditional router. This allows you to play with LLDP, BGP, OSPF, etc. I won’t go into details of a setup like this for this article, and is left as exercise to the reader.
In the previous scenario, using one Appliance was nice, but how about using multiple appliances together? Like creating a virtual CLOS? For that we will be using GNS3.
If you never heard of GNS3 before, is an Open Source platform to create virtual networks and deployments inside your laptop, with some nice UI. Even more useful, it’s cross platform. I will be focusing my demo here on using it over Mac, but should be easy to extrapolate if you are a Windows user.
In order to use the appliance with GNS3, you need to import the OVA file into VirtualBox (File -> Import Appliance). The OVA generated by OpenSwitch is compatible out of the box with GNS3 since April 26th 2016.
First, download GNS3 from their project page and install it (yes, you have to register). Next, I recommend that you install Wireshark to have the capabilities of snooping on the traffic of the virtual network.
On recent OS X, when trying to run the GNS3 application you may get an error message:
“GNS3” is damaged and can’t be opened
This is a wrong error message, and the problem is really that you need to reduce the security of the system to allow executing applications that are not from the App Store. You can change this setting on: System Preferences -> Security & Privacy -> Allow apps download from: Anywhere.
When you first start GNS3, it will start the Setup Wizard, and ask for a server. Select the ‘Local server’. We would not need the GNS3 VM for running the OpenSwitch appliance, since will be run on top of VirtualBox.
On the next window select to ‘Add a VirtualBox virtual machine’ (at this point you should have already imported the OpenSwitch Appliance into VirtualBox for this to work).
Next, select the OpenSwitch Appliance Image that was imported into VirtualBox, and enable “Use as linked base VM”.
After of click on ‘Finish’, the preference panel for VirtualBox VMs will show and you will have to edit the recently created template to add some custom configuration: On the ‘General Settings’ tab:
On the ‘Network’ tab:
Now, it’s time to deploy two instances and verify connectivity between them:
At the end of this setup your project should look like this:
Before starting the simulation, I will attach wireshark to one of the interfaces to see the packets between the machines:
Now you can start your machines by hitting the top ‘Play’ button. This will start both VMs. Then you can double click on both machines and a Terminal will open where you get access to the serial console of the machine. You can login with the user and password ‘netop’ (this will enter into the traditional CLI mode, but you can also login as root into a bash shell):
At this point you can right click on the link where we are capturing the traffic, and select ‘Start Wireshark’. This will open a WireShark window that is feeding from the pcap capture of the traffic from the port.
Note: There is currently an open bug in OpenSwitch where the image may fail to boot properly the data path. To validate your appliance booted properly run the following command:
1
|
|
If the reported MAC address is (null), then reboot the appliance and check again. I will update this blog as soon as the bug is closed.
Now is time to see some traffic flowing between the machines: let’s enable LLDP on both ports, and we should be able to see both switches identifying each other, and the traffic flowing thru Wireshark. Let’s issue the following commands on the CLI of both switches:
1 2 3 4 5 6 |
|
After these commands are entered on both switches, we can now show the LLDP neighbor information on any of the switches:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
And you can see the LLDP advertisements going thru with Wireshark:
That is all the fun that we will have for today, hope you find the information useful. I’m planing to blog soon about the following subjects:
Stay tuned!
]]>Note: This article was originally published here.
One of the purposes when we designed the build system in OpenSwitch, was to make it possible to develop on as many environments as possible. If you have some background with developing networking firmware, the typical developer love to have this VM where everything works perfectly, but makes it impossible to work in your laptop at 30000 feet. This is not really a sin (as long as you can have the VM hosted in your machine), but the problem is that usually is some IT team on charge of the VMs setup, and the deployment is not handled by some automated/version-controlled code.
So for OpenSwitch, we aimed to at least document the requirements and steps for manual setup of your environment. You can read this page to get your Linux machine to ready it for OpenSwitch development.
So, why to write an article about my particular setup? Well, I’m a Mac user, so in this article I’m going to detail my setup using a OS X host with a Linux VM. This provides some nice tricks that makes your workflow easier if you are using a similar setup. I will also explain the rationale of the setup.
My use of Linux VM for development is mostly thru the Linux CLI and I use NFS to share files between my Mac and Linux. This allows me to use any graphical tool from the Linux VM if I have to, but also to use tools from the host without major hurdles.Read more…
The first step is that I will use VirtualBox to deploy my Linux VM. You can use any software that you want, but VirtualBox is free and available to any developer, so it’s a good target.
Since I’m using a Macbook Pro Retina with a i7, I will use the following setup:
For this VM, I will be using Ubuntu LTS 14.04 Server ISO. Right now OpenSwitch works best with this version of Ubuntu (and provides faster build times). We hope to update to the new LTS soon and once we do, I will update this post to reflect that. Update: I have added comments about how to make it work with Ubuntu 16.04!.
I’m assuming you are familiar with installing an Ubuntu ISO into a VM and will omit those details. The only important detail from the installation would be to install the SSH server. I would recommend that you setup your user account with the same user id that you have in GitHub (this is a good moment to create one, if you don’t). Using the same user account makes life easier with gerrit, but is not an strict requirement.
If you are behind a corporate firewall, be sure to setup your corporate proxies in your user environment:
1 2 3 4 5 |
|
If my case, I may be roaming between the office and home, so sometimes I don’t use a proxy, in which case I find easy to unset the variables and be able to work. So, I don’t like to wire the apt configuration with the proxy as explained here, instead I like to run the apt-get command with ‘sudo -E’ parameter so that it inherits my environment variables and uses the proxy if I have it defined.
First, I usually disable asking password for my user:
1
|
|
Then enter a line like the following (replace your username):
1
|
|
Next, let’s update the apt package cache:
1
|
|
The initial installation will configure the machine out of the box to have DHCP on the first network interface, so we will need to add the second interface for automatic startup. So I will add the following lines to /etc/network/interfaces:
1 2 |
|
Then, bring the interface up:
1
|
|
Update: if you are using 16.04, the network interfaces have different names (to understand why and revert it if you want, take a look here). Run ‘ifconfig -a’ to find the name of your interfaces.
Next, proceed to install some extra packages and configurations that make networking work better.
dnsmasq: the build system fetches constantly from the servers in order to use the shared states to speed the build. So using a local DNS cache will help reduce the pressure on your DNS (and may save you from raising flags about the high rate of request your machine will be doing).
1
|
|
Update: if you are using 16.04, dnsmasq won’t work out of the box since it conflicts with the dnsmasq used by the LXC container daemon. I worked around by disabling LXC (but maybe that solution doesn’t work for you):
1 2 |
|
nss-mdns: this is the name service resolution to use multicast DNS. This allows the Linux machine to resolve (and as side effect of dependencies, advertise) .local addresses and makes communication with the Mac simpler.
1
|
|
After this is installed, you should be able to reach your machine with the .local name. Lets say your Linux machine is named linuxvm, then you should be able to run from your Mac:
1
|
|
After the initial networking setup is ready, I like to run the VM in headless mode from the OS X terminal. To do that, I shutdown the VM from the VirtualBox UI and close the application. Then from an OS X Terminal I can start the VM in the background with (replace the LinuxVM name with the name of your VM:
1
|
|
This will run on the background and as long as you don’t close the terminal, the VM will run. Next, I usually connect to the VM from the terminal with (using the mDNS .local resolution):
1
|
|
The -Y automatically exports X11 apps into my Mac XQuartz, in case you want to use some graphical program from Linux.
Create a workspace directory on /ws. This is just an standard I follow of putting all my work on a separate directory. Makes easier to fine-control the NFS exports from my Linux machine into my Mac host.
1 2 |
|
Installing an NFS server with the proper parameters to allow easy sharing of files between the host an the Mac took me several hours of research. Here is the recipe for it.
Install the NFS server daemon:
1
|
|
Configure the /etc/exports file with the following line to allow your Mac host to access the files under /ws with the same user permissions as your Linux user (assuming your user ID is 1000, you can find that with the ‘id’ command).
1
|
|
Start the NFS server:
1
|
|
Update: if you are using 16.04 use systemctl instead of service, and the service is called nfs-server:
1
|
|
At this point you should be able to mount the NFS directory from your Mac, by going to the Finder Menu Go -> Connect to Server… (or using the ⌘k shortcut). On the server address box enter:
1
|
|
That should get your Mac access over NFS to the directory /ws and will have the right permission mapping so that files could be changed either from the Linux or Mac environment without permissions conflicts.
We are going to be using docker, so we need to install it following the instructions here.
If you are using a proxy, you need to setup it up for the docker daemon on /etc/default/docker:
1 2 |
|
Then, restart your docker to make the changes take effect:
1
|
|
Install the minimal required packages for OpenSwitch development:
1
|
|
OpenSwitch leverages Yocto’s shared-state system, so we want to have a global shared state directory that is shared across all the development environments to speed build time. The OpenSwitch build system will get this information automatically from a variable environment, so I will put it on my profile by default:
1 2 3 4 |
|
Setup some initial git setup for working with OpenSwitch Gerrit. You will need a GitHub account in order to login into gerrit, so I will assume you already have one, and you know how to upload your ssh key into gerrit. You will also need to sign a DCO when signing for Gerrit, otherwise won’t be able to upload code for review (if you are going to contribute as part of your job, you need to follow the steps on the OpenSwitch page to get your employer to sing a corporate contribution agreement).
First, setup git (I like to use colors, and use st as an alias for status):
1 2 3 4 |
|
If you are working behind a corporate firewall, it may be that gerrit ports (29418) are blocked. You can test a telnet connection into the port to confirm if you are blocked. A workaround this restriction is to install socat and use a proxy for the ssh connection:
1 2 3 4 5 6 |
|
If your github username is different from the username in your machine, you may want to configure git to use the right username when doing git reviews:
1
|
|
You can verify your connection to gerrit works by issuing this command and getting a similar output:
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Finally, I’m going to clone the OpenSwitch build system, and build an appliance image that I would use in my next post about working OpenSwitch with GNS3.
First I clone the build system:
1 2 3 4 |
|
Now you can sit a wait for the build to complete. If you have a fast internet connection it will take in the order of 15 minutes to download all the caches and sources to build a full OpenSwitch Virtual Appliance. In my next post we will be using that.
I hope the information is useful. Drop me a comment if you think there is any improvement I can do to the environment.
]]>You see, I’m a software guy. The pragmatical Linux/OpenSource fanboy kind. What that means? I have a formal degree on Computer Science, and wrote Linux drivers and software for embedded systems for 8 years. But I’m also a pragmatical guy: I know how to write kernel drivers, but I use a Mac laptop every day because I like things to work. For the last 4 years I have been learning a big from networking at Hewlett Packard Enterprise, where I have worked on networking (SDN, ASICs), and more recently on the OpenSwitch project.
Why start writing about the intersection of Open and Networking? These are interesting times in the networking world: disaggregation, open source, open hardware, etc. All of this is pretty new for the networking world, and is specially different because in the old good days you will have limited information about the technical details of what is going on behind scenes on your network. Now sharing the knowledge is critical. I have been inspired by reading blog entries from many in the networking community, and I’m by no means close to have the expertise of guys like Ivan or Greg, but I think I can contribute to spreading the knowledge on the little things that I do know. Hope my readers agree.
Stay tuned for some posts soon. I will be writing quite a bit about OpenSwitch, but also random thoughts on different areas about Open + Network.
]]>