BIG shot out to VMWare ESXi, because that finally did the trick. Currently, I'm visiting family for the holidays but I still want to be able to study when I can. I know that's super nerdy, but if you're reading this... you're not much better than me. So stop judging. I've recently purchased a server off eBay (nice Dell R610 with 32GB of DDR3 for ~300 bucks), but it's not yet arrived so I've been trying to get something workable in place for the interim. At home my workstation has 32GB of RAM, and is fairly capable of running these SP labs. However, I'm getting both off point and a little ahead of myself. Let me first tell you about my lab topology, from INE's SPv4 material.
- (10) CSR1000v. The CSR1kv has certainly come a long way in terms of requirements, but it's still a hefty beast. You can technically get by on (1) vCPU and 2.5GB of RAM, but I like to run mine with 3GB of RAM. So right off the bat, I'm at a potential 30GB of RAM needed.
- (4) IOS-XRv. IOS-XRv runs perfectly well with only 2GB of RAM, so we're up to a potential 38GB of RAM needed.
So, obviously these VMs will rarely (if ever) use that much memory. That doesn't matter with VIRL though, because it will still prevent you from firing up that environment without 38GB of available memory. First attempt at resolving this, lie to VIRL. I went into my VMWare Workstation settings and told it "Allow most virtual machine memory t be swapped"
However, for this to really work I'd need to limit overall memory usage of all my VMs to ~8-10GB and swap the rest. Workstation doesn't like that, anything over 3/4 being swapped and Workstation will not let you boot the VM. Which wouldn't be a huge issue, except for the added overhead of VIRL itself, requiring anywhere from 2.5 - 5GB of RAM to run. Also keep in mind, I know that swapping that much will just murder performance, but I'm not after performance. I'm after boot 14 routers that move JUST fast enough that I can lab with them until I get home. So I hit a wall there. I tried my GNS3 Server approach (as covered in earlier posts)... but GNS3 can have odd issues with environments that get powered off and on several times. I even ran into issues with shutting down the server with GNS3 still open... then the server would just be perpetually hosed upon next boot. This just felt like WAY too much effort lol.
"Okay, we get it. How did you actually get it working?!"
I had this half baked idea, that I honestly felt wouldn't work. "What if I nest ESXi inside workstation? Give my ESXi VM a small amount of memory, then just try to boot everything?" I know ESXi is way better about swap when it runs out of RAM (hence why I bought server explicitly for my SP lab), but I didn't think this would work well enough to actually use. I was wrong. I gave ESXi 8GB of RAM, loaded up (10) CSR1KVs and (4) IOS-XRvs and booted everything up. My laptop's CPU spiked really hard, and remained at ~89% for 5-10 minutes. After that though, it seemed to settle down well enough, and... everything booted. More than that, I copied base configs in for all devices and everything was talking!! Response times between devices is fairly high, but no more than dynamips routers really (averaging 30-60msec). That's totally ok though, and the console response is almost perfect after everything is booted. Nearly no lag... enter screen shots!!!
As you can see, all nodes booted and ESXi is just amazing at memory management apparently. On my host side, here's what task manager looks like:
I mean, it's definitely loaded (and when my work VM is running too, mem is at 96% utilized), but my laptop isn't melting!! As a follow up to this post, I might record a video showing off my VMWare config, the full on home lab I'm planning on using vSphere so I can have easier packet captures with a jump host. That's all for now though! Thanks for reading, and comment if you have questions or suggestions.