(Upcoming) Solaris Features: Crossbow - Part 2: Limiting and Accounting
One of the basic objects in the new Crossbow stack is the flow. Any network traffic is separated into such flows. And with this flows you can do several interesting things. In this article i want to present two usages of them: Bandwidth Limiting and Flow Accounting
Demo environment
I did the demonstration in a simple test environment. a340 is workstation under my desk connected with Gigabit Ethernet to an Airport Extreme (AE) in bridging mode. The system has the ip address 192.168.178.109 and works as a server in this demo. It’s a basic OpenSolaris 2009.06 installation with installed apache22-packages. a330 is a notebook connected via 802.11n to the same AE and it’s used as the client.
Bandwidth Limiting
Of course, most of the times you want to transport data as fast as possible. But there are situations, where you want to limit the amount of network traffic. Let’s assume you provider shared hosting on a platform and you want to sell certain service levels. For example a service level with unlimited bandwidth, one with 2 MBit/s per second and one with 8 MBit/s. If you don´t have any mechanism to limit the bandwidth, anybody would just order the 2 MBit/s service as she or he get unlimited bandwidth in any case.
Let’s measure the unlimited traffic at first to have a baseline for testing the limited transmissions.
jmoekamp@a330:/tmp$ curl -o test1 http://192.168.178.109/random.bin
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 5598k 100 5598k 0 0 6433k 0 --:--:-- --:--:-- --:--:-- 6464kAs you see we are able to download the data 6464 Kilobyte per second. Okay, let us impose a limit for the http server. At first we create a flow that matches on webserver traffic.
jmoekamp@a340:~# flowadm add-flow -l e1000g0 -a transport=tcp,local_port=80 httpflowWhen you dissect this flow configuration you get to the following ruleset:
- the traffic is on the ethernet interface
e1000g0 - it is tcp traffic
- the local port is 80
- for future reference the flow is called
httpflow
With flowadm show-flow we can check the current configuration of flows on our system.
jmoekamp@a340:~# flowadm show-flow
FLOW LINK IPADDR PROTO PORT DSFLD
httpflow e1000g0 -- tcp 80 --This is just the creation of the flow. To enable the bandwidth limiting we have to set some properties on this flow. To limit the traffic we have to use the maxbw property. For our first test, we set it to 2 Megabit/s:
jmoekamp@a340:~# flowadm set-flowprop -p maxbw=2m httpflow A quick check, if we did everything correct:
jmoekamp@a340:~# flowadm show-flowprop
FLOW PROPERTY VALUE DEFAULT POSSIBLE
httpflow maxbw 2 -- 2m
httpflow priority -- -- Now i use my laptop as a test client and download the file again:
jmoekamp@a330:/tmp$ curl -o test1 http://192.168.178.109/random.bin
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 5598k 100 5598k 0 0 265k 0 0:00:21 0:00:21 --:--:-- 266kAs you see … 266 Kilobyte per second that’s, roughly 2 MBit/s. Okay, now we try 8 Megabit/s as a limit:
jmoekamp@a340:~# flowadm set-flowprop -p maxbw=8m httpflowWe check again for the properties of the httpflow
jmoekamp@a340:~# flowadm show-flowprop
FLOW PROPERTY VALUE DEFAULT POSSIBLE
httpflow maxbw 8 -- 8m
httpflow priority -- -- Okay, a quick test again:
jmoekamp@a330:/tmp$ curl -o test1 http://192.168.178.109/random.bin
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 5598k 100 5598k 0 0 936k 0 0:00:05 0:00:05 --:--:-- 933kOkay, we yield 933 Kilobyte/s. That’s a little bit less than 8 Mbit/s
Accounting
Okay, all the traffic in Crossbow is separated in flows (when it’s not part of a configured flow, it’s part of the default flow). It would be nice to use this flow information for accounting. Before doing the testing i activated the accounting with the following command line:
jmoekamp@a340:~# acctadm -e extended -f /var/log/net.log netNow i can check for bandwidth usage. For example when i want to know the traffic usage between 18:20 and 18:24 on June 20th 2009 i can use the flowadm show-usage account you yield this data from the file i’ve configured before (in m case /var/log/net.log:
jmoekamp@a340:~# flowadm show-usage -s 06/20/2009,18:20:00 \
-e 06/20/2009,18:24:00 -f /var/log/net.log
FLOW START END RBYTES OBYTES BANDWIDTH
httpflow 18:20:27 18:20:47 0 0 0 Mbps
httpflow 18:20:47 18:21:07 0 0 0 Mbps
httpflow 18:21:07 18:21:27 104814 6010271 2.446 Mbp
httpflow 18:21:27 18:21:47 0 0 0 Mbps
httpflow 18:21:47 18:22:07 0 0 0 Mbps
httpflow 18:22:07 18:22:27 0 0 0 Mbps
httpflow 18:22:27 18:22:47 0 0 0 Mbps
httpflow 18:22:47 18:23:07 0 0 0 Mbps
httpflow 18:23:07 18:23:27 121410 5333673 2.182 Mbp
httpflow 18:23:27 18:23:47 15246 676598 0.276 MbpsThe capability to do accounting on a per flow basis makes this feature really interesting even when you don’t want to configure a traffic limit. So i configured an additional flow for SMTP traffic and now the accounting is capable to separate between the HTTP and the SMTP traffic:
jmoekamp@a340:~# flowadm show-flow -s
FLOW IPACKETS RBYTES IERRORS OPACKETS OBYTES OERRORS
httpflow 1168 77256 0 4204 6010271 0
smtpflow 18 1302 0 13 1250 0Do you want to learn more?
man-pages
flowadm (1m)
Others
opensolaris.org: Crossbow project
Crossbow: From Hardware Virtualized NICs to Virtualized Networks