Cisco Ios Xrv 9000 Gns3

 

The claims and recommendations in this post are my experiences experimenting with the different software involved over a weekend. I could very well be wrong about multiple things here, and if you feel that I am, don’t be afraid to comment.

Software used:

Others like Cisco do not. You will be need to provide your own Cisco images to use Cisco devices with GNS3. GNS3 are unable to provide Cisco IOS images. Please do not ask us or others to give you an image. You will need to either have an authorized account with Cisco or purchase a VIRL license to access Cisco images like the Cisco ASAv or IOSv. GNS3 VM is recommended for most of serious appliances (IOSvL2, IOU, IOSv, IOS-XRv, ASAv, Arista) as local windows machine would struggle. F5 appliances will run on VM only. I would consider server. Looks like you're using an older browser. To get the best experience, please upgrade. UPGRADE MY BROWSER.

  • Windows 7
  • GNS3 1.1
  • VMware Workstation 10.0.1
  • Virtualbox 4.3.18
  • XRv 5.2.0

1. Introduction

Cisco’s XRv is a VM of their IOS XR operating system that can be deployed on x86 systems. Commercially, it’s marketed for use as an out of band route reflector, but for enthusiasts like us, it’s a tool for playing around with XR without needing to rent relatively expensive rack time, or buying routers instead of a new car. It doesn’t have a complete set of feature, and lower layer features that are “closer to the hardware” are less likely to work. An example would be something like VPLS. This thread over at one of Cisco’s forums has more discussion on that issue: https://supportforums.cisco.com/discussion/12101386/xrv-511-workingnon-working-features-and-roadmap

XRv is supposed to be deployed on a bare metal hypervisor which presents a problem for home users like me who only has a single computer with enough RAM. The good news is that it works fine on VMware Workstation. The bad news is that if you don’t read a post like this one, you risk losing a ton of time on solutions that should work, but really don’t. A big problem with XRv is its boot time of 10+ minutes which makes experimenting with different set ups incredibly tedious.

This post will not describe every single detail of these configurations and will instead simply point you in the right direction to avoid the worst pitfalls. If you need more details, I’m sure you can find that elsewhere.

2. What Doesn’t Work

2.1 GNS3 + Virtualbox + XRv

Any kind of integration with GNS3 seems to be broken. What’s worse is that it’s not completely broken, occasionally allowing packets to flow – something that caused me to waste a lot of time on it. The general idea on how to get XRv into GSN3 is:

  • Install Virtualbox
  • Create one or more XRv VMs in Virtualbox and assign them several network adapters connected to host network adapters.
  • Each installed network adapter will map to one out of 4 “physical” interfaces within XRv
  • Install GNS3
  • Use GNS3’s Virtualbox wrapper feature to import the VM into GNS3.
  • Connect the VM to whatever else you have in your GNS3 topology.

The problems are twofold:

  • The interface numbers do not properly match up. Meaning, the first physical interface in XRv will not always be interface 1 in GNS3. This wouldn’t necessarily be a problem if it was a consistent offset, but it doesn’t seem to be.
  • Worse, when I imported two identical sets of VMs, communication was possible between two of them, but the other two were dead. This is some kind of bug with GNS3, Virtualbox and XRv. This long thread has more details: http://forum.gns3.net/topic8394.html
  • The solutions presented in that thread involve things like removing and replacing links between devices, something that you wouldn’t be interested in messing around with when labbing.

2.2 GNS3’s Cloud Feature

The next thing I tried was using GNS3’s cloud feature. For anyone not familiar with GNS3, it essentially allows you to send packets from any network adapter installed on the machine into whatever device you have it connected to in GNS3. It’s great for things like connecting Windows or Linux VMs to your router topology. I had high hopes for this approach because both Virtualbox and VMware Workstation allows you to output traffic from a “vswitch” into a software network adapter installed on the host system. Getting the traffic from the “virtualized” NIC into GNS3 through the cloud feature seems conceptually viable, but I could not get it to work. With Virtualbox I had a similar experience to how Virtualbox behaved with the previously mentioned wrapper feature; it worked, and then it didn’t. VMware Workstation didn’t want to play ball at all.

3. What Actually Works

Using VMware Workstation (VMW), but keeping all traffic within the virtual network of that system seems to work without issues. In other words, your devices will communicate with one another using VMW’s virtual switches, called vmnets. According to VMware, each vmnet is a “virtual switch”, but it’s conceptually more like a virtual hub. When a networking person thinks of a switch they think of a device with ports where each port belongs to certain vlans and that the switch learns the location of MAC addresses connected to other ports that belong to the same vlan, but this doesn’t seem to be how a vmnet work. Instead, it appears that frames are simply copied to all other adapters connected to the same vmnet, irrespective of what vlan-ids you add the the frames.

Normally, a switch would drop frames tagged with a vlan-id if the port expects untagged frames, but a vmnet forwards tagged frames just fine. Another way of thinking about it is that the vmnet is a switch where all ports are trunks with an allowed list of 1-4095. Ultimately, it’s hard to verify any of this beyond simply observing how connected devices behave since you don’t have any visibility into the vmnets. At first glance, this might seem like a bad thing, but it’s actually not since it allows us to create arbitrary topologies with just one vmnet. More on that later.

3.1 Configuring VMW For XRv

3.1.1 Creating the VM

Creating an XRv VM just involves adding a new VM with the XRv .vmdk as the harddrive. It’s supposed to have 3 GB of RAM, but I’ve heard that you can get away with less. Downloading the XRv files through Cisco’s normal software channels requires an account with certain credentials, however, using this link: https://upload.cisco.com/cgi-bin/swc/fileexg/main.cgi?CONTYPES=Cisco-IOS-XRv allows a normal user like me to download it. Presumably, this discrepancy is a mistake, but who knows.

3.1.2 Getting Console Access

After successfully creating your VM and booting it, you’ll eventually end up with a blank screen. It’s easy to think that you’ve broken something, but that’s not the case. The router console output is just not displayed in the standard VM display output. To fix this you need to use something called a serial named pipe. To get console access you need to add a serial port to the VM and set it as named pipe:

CiscoCisco IOS-XR: Deploy an IOS-XRv appliance in GNS3 – Just Lab ...


If you are like me, you’ve never heard of a named pipe before. What is it? It’s basically a way to copy (or “pipe”) any input/output to and from the virtual machine’s serial port from the host system.

To interface with the named pipe via telnet, we need to use a tool called Named Pipe TCP Proxy found here: http://shvechkov.tripod.com/nptp.html. After installation it should be self-explanatory how to add a named pipe and map it to a port on the localhost. Just note that you need to start the program as administrator for it to be able to save settings.

At this point you should have console access to your XRv VMs,

3.1.3 Network Adapters

At this point your XRv VM will not have any external facing interfaces because you haven’t properly enabled any vNICs for it. The first network adapter added to the VM will always be the management interface, and any subsequent adapters will map to Ethernet interfaces on the XRv. This image, which I’ve borrowed from Cisco’s XRv configuration guide, illustrates this:

We will not be using the mgmt interface since we have our named pipes for console, but we do need at least 1 Ethernet interface. The minimum is thus 2 network adapters on the VM to get access to the first gig0/0/0/0 interface. According to the configuration guide, XRv supports up to 128 interfaces.

3.1.4 Editing the .vmx file

Just adding the network adapters to the VM will not work since they, by default, will be of the wrong “type”. XRv wants the “e1000” type, and this must be manually edited in the VM configuration file since there’s no option in the Workstation GUI. Find the .vmx file and open it with a text editor. You should see something similar to this:

Ios Xrv

[…]
ethernet0.present = “TRUE”
ethernet0.connectionType = “custom”
[…]

Add the line ethernet0.virtualDev = “e1000” after ethernet0.present:

[…]
ethernet0.present = “TRUE”
ethernet0.virtualDev = “e1000”
ethernet0.connectionType = “custom”
[…]

Do this for all adapters that you’ve added to the VM and they should show up as external facing interfaces in XRv.

4. Networking

You could create a network by adding several interfaces to each VM and connect them to different vmnets to create a topology. This is quite inefficient however since creating a new logical topology would involve reconfiguration of the vmnets. It’s easier to connect one interface from each XRv to the same vmnet and rely on dot1q subinterfaces to create the logical topology.

Creating any arbitrary connection now just involves adding a dot1q subinterface on the two devices in question.

5. Conclusions

I hope that this post has been helpful and that I haven’t left out any key piece of information. Good luck!

Beginning of this year, i learned about the Nexus 9000V and I immediately saw the chance to use it for the CCIE DC preparation (and for some programming tests as well, but that’s for a future blogpost). The 9kV is basically a virtual Nexus 9300 that you can run on ESXi, Virtual BOX and KVM (the qcow2 Version is that one you need for GNS3.

Here are the pro’s and con’s for the 9000V, because as always, where’s an upside, there’s also a downside.

Pros:

  • Free for testing/training use (you do need a 9k Contract to get it from cisco.com)
  • You can simulate a lot, even VPC (since NX-OS 7.0(3)I5(2)) and much more … 😉
  • Also there are a lot of untested features, that “might” work (just have a look into the config guide)

Cons

  • No VXLan EVPN Support (but VXLan without EVPN should work)
  • You need power to run it, 8GB Ram and at least 1-2 CPUs (Cores) should be Available.
  • A few tweaks are necessary to get it running in GNS3.
  • Traffic through the 9kV is limited to 4mbit, but hey it’s a lab …

Here is the Link to the Configuration Guide, where you can Lookup all details: NX-OSv 9000 Guide

I don’t replicate the whole step for installing it, since Andras Dosztal does have a great guide on his blog to get you started: How to add NX-OSv 9000 to GNS3

Once you’ve done that, there are a few minor stumbling blocks:

  • The Nexus will boot the first time, but on the second it won’t find the image and is ending up in the loader. This is how to fix that:

config t
boot nxos nxos.7.0.3.I5.2.bin
copy running-config startup-config

  • Be sure to have enough RAM and CPU’s if you boot them at the same time. I had really strange behaviors before i granted every 9kv at least 8GB and 1CPU.
  • Another major thing is , some L2 Commands as “show mac address-table” are different. If you use the regular NX-OS you get wrong infos (you don’t see MAC addresses that are there)

So what is my first impression after spending a weekend playing around? The Nexus 9000V is an ABSOLUTELY great thing. It has it’s limitations, but for prepping for the EXAM i think this will help me out a lot. Of course it isn’t a full replacement and you have to be careful what’s working and what’s not. But that’s to find out in the next weeks.

Ios Xrv 9000

So if you want to know more about the lab on the drawing? Stay tuned for the next Posting here! Just so much 80GB Ram, 16 CPU’s are a pretty good start, but I’m testing to how much i can limit the Ram, just so much 32GB are definitely too less. 🙂