The individual owning this blog works for Oracle in Germany. The opinions expressed here are his own, are not necessarily reviewed in advance by anyone but the individual author, and neither Oracle nor any other party necessarily agrees with them.
Wednesday, December 28. 2011
Work in Progress - this entry will change often in the next days and weeks
A few days^H^H^H^Hweeks ago, i wrote about simulating the cloud that is most often tagged with the name "network" or "intranet" and sometimes "internet" . This would not be c0t0d0s0.org without an article to explain how you can configure this. This article will explain how you simulate a complete network on a single host with routers, switches, dynamic routing protocols and so on
At first i want to set the expectations right. I don't want to simulate a cloud in the sense of cloud computing here. I'm thinking about something more complex:
I'm talking about the simulation of this cloud, that often hides a lot of complexities and traps in architectural diagrams.
A word of caution first
This article uses a invisible feature. You don't see that it's there because it isn't in the man page, it isn't in the help output of the
Why i'm writing about this "feature" here? Because it's useful. Because there are a multitude of hints that this function exists. All of them are public. The zonestat documentation mentions a "simnet" type at docs.oracle.com and from there you are just a google away from the PSARC case 2009/200. And the source code at src.opensolaris.org shows it as well. From there it's just curiosity to find everything else out that is used in this text.
About this article
I stumbled the first time over this command when i searched for something in the dladm source at src.opensolaris.org. A month ago my former colleague Brian Utterback remembered me of this and i though "let's check if this is still working". And to my astonishment it still worked.
Writing this article takes virtually forever. Because of my broken ankle i took painkillers and that made me somewhat drownsy. And this drownsyness slowed down everything. Thus i decided to create this article under your observation to get it finally out of the door. Thus it's work in progress.
I just write about simnet. What are simnets? I just want to point you to the PSARC case for indepth information. It's available on opensolaris.org in the caselog. But in short: Simnets are simulated networks. It's a mechanism to test networking protocols. And in this example we will use it exactly for this purpose. Testing networking.
Okay, let's assume you are admin of FUBAR Inc. You want to recreate your network in a box. You have offices in Hamburg, London, Singapore, New York and San Francisco. In each office you have a multi-legged router, connecting to a switch for the internal network with servers an clients, the other interfaces of the switch are connecting to the other routers. As an image says more than 1000 words i will just summarize the network with this figure.
Of course the and the servers will be zones. However we have to recreate the network topology as well. And that's the point where we use the the simnet non-feature.
We need a the switches in our offices first. Those are really easy to configure
Now i need some switchports. At first i create some switch ports in order to connect the switch to the router.
Now i create some additional switchports to connect servers.
Ports meant for the bridge are nice, however they should be connected with the bridge.
Let's now create all the interfaces we need for the routers.
And of course we need interfaces for all the servers
Now we have to create logical cables … lots of them. At first the routers with their switches.
Uff … on the networking side this is all.
The active configuration should look something like that ...
Okay, now we have to create the zones.
We create a lot of controlfiles first. With this controlfiles we will feed
Whois is wondering about the sfo and sin IATA shorthands that i've used instead of the long names in other "cities". Quagga doesn't seem to like interface names longer than 16 characters.
Okay. Now we have to create all the zones. That's easy. As i said, i will just feed the control files into
Okay, at first we install the template zone. We do a full install here. and that's pretty much the only purpose … to have one installed baseline zone as providing the starting point for all other zones. This may take a while. Depending on your system you may opt for a coffee or two.
We never boot this one, it's just to ease the next steps.
Okay, now we prepare the real zones. You don't have to to the next steps, however they relief you from login into each zones and going to the same dialog windows. We will use a simple trick to circumvent the need to go through each sysconfig dialog in each router we will use a simple trick. You can create a xml file containing the necessary data and pass it to the cloning of the zone.
Important: I want to make the resulting xml file as generic as possible, thus i won't configure networking via this process, albeit this is possible. As it's a CUI, i will guide you through this dialog with some pictures.
After leaving the last screen, you should yield a file with content similar to this:
Before you ask, the password for radmin and root is n0mn0mn0m. And the jamphfhn just stands for "just a meaningless placeholder for hostname".
Okay, i will create another template zone. This is because a routing zone will have some special properties that a zone acting as a server doesn't need and i don't want such properties in the server zones.
At first i just take the template.xml script and substitute the hostname. I could simply do it via vi, but for a tutorial a simple shell line is more efficient.
I use the newly created file as an input for the zone clone command.
As the system just creates a zfs clone the command should return after a small period of time. Now we can log into the console of the zone with zlogin.
I wrote earlier, that the template for the router contains some additional stuff. At first i need a telnet client. It will get obvious why i need it later on:
Okay, now let's install quagga. Quagga is a suite of daemons to implement dynamic routing protocols:
Okay, now we have to configure some basics that are equal to all the router in the network.
At first we activate forwarding. With this activation, you enable the operating system to accept packets on one interface
Okay, now we have to do some quagga configurations. I want to use quagga with OSPF, so there are two important services for me. Zebra and ospf. Zebra is the layer, that the quagga suite used to interact with the system. Why is it called Zebra? I assume it's history, the old GNU routing protocol daemon suite was called zebra, quagga is the follow-on project as zebra is now a defunct software development project. What do we configure here.
Both daemons offer a command line for interfaction with the daemon. We configure both just to react from 127.0.0.1 (aka localhost). The zebra daemon has it's console on port 2602, the ospf daemon listens on port 2601. And this both ports are the reason we need telnet on our routers. You access the consoles via telnet.
With this command we tell Solaris to use ospf as the routing protocol for ipv4 purposes.
Now we have to activate the new setting
You should now get some weired SMF error messages that some services couldn't start up. that's normal because there are no configuration files available for the quagga suite. Don't think about it, just shut the zone down now.
Okay, now we have derived our template for the router zones from the generic template for zones. We use this template for installing all the router zones now.
Okay, i just wrote about quagga config files. I want to prepare them now in order to be able just to copy them into the zones before starting them up and thus to circumvent the error messages. We need a lot of them.
Put something like this into the file
Switches for Hamburg-MAN
There is a problem. When you closely look at the
Display comments as (Linear | Threaded)
Okay, so you knew about simnets (faceplam).
One question, why use "create-bridge" instead of "create-etherstub"?
#1 Brian Utterback on 2011-12-27 19:10
Actually the face palm goes on me ... you was the person remembering me of simnets, not Matt ....
#1.1 Joerg Moellenkamp on 2011-12-27 19:18
AFAIK etherstubs just look like interfaces connected by a switch, whereas bridges are really switches. Switches have their own set of problems (like the suboptimal spanning tree at the end in "London" when using defaul STP configuration). Furthermore you can't play around with TRILL with etherstubs.
Lovely. Would the add-in hxbt driver be in scope for this exercise?
That would really enhance the WAN simulation IMHO. -cheers, CSB
For those who aren't familiar, check out:
Of course ... but so far i hadn't time to configure this into my testbed ...
I'd like to say it's a great article that I can't keep myself from trying this in my lab server.
There're some issues when I followed your example. I guess you missed them careless.
1. You didn't give out /opt/cloudsimulation/zones/templaterouter zonecfg file, I guess it should be like this one
2. You didn't give out /opt/cloudsimulation/zones/londonsrv1 which was also used to create newyorksrv1.
I guess it should be like this one
3. When create-ip in newyorkrouter zone, there's a wrong ip provided.
ipadm create-addr -T static -a 10.0.14.0/24 newyorkrouter0/v4
ipadm create-addr -T static -a 10.0.14.254/24 newyorkrouter0/v4
I figured out the address from the network1.png
Again, it's a great article and good for exercise.
Well ... one of the reasons why this is still work in progress
#3.1 Joerg M. on 2011-12-31 21:49
The author does not allow comments to this entry
The LKSF book
The book with the consolidated Less known Solaris Tutorials is available for download here
Martin about End of c0t0d0s0.org
Mon, 01.05.2017 11:21
Thank you for many interesting blog posts. Good luck with al l new endeavours!
Hosam about End of c0t0d0s0.org
Mon, 01.05.2017 08:58
Joerg Moellenkamp about tar -x and NFS - or: The devil in the details
Fri, 28.04.2017 13:47
At least with ZFS this isn't c orrect. A rmdir for example do esn't trigger a zil_commit, as long as you don't speci [...]
Thu, 27.04.2017 22:31
You say: "The following dat a modifying procedures are syn chronous: WRITE (with stable f lag set to FILE_SYNC), C [...]