I already used this feature in the HA load-balancing tutorial. However, the feature is too useful to remain just a side note in another article. It is DLMP, or by its full name: Data Link Multipathing.
Differences
DLMP is a somewhat distant relative of IPMP, but it works on a different layer of the network stack. Whereas IPMP is purely IP-related and thus operates on IP interfaces, DLMP works on the data links. For this reason you can do a lot more with it — for example, setting up VRRP on top of it. (Because VRRP manipulates MAC addresses, it doesn’t work on IPMP interfaces.)
DLMP is a closer relative of trunk aggregation, also known as IEEE 802.1ax or 802.3ad (EtherChannel and LACP are other related names for the same or similar things). I will call this type of aggregation “trunk aggregation” in this article.
You configure a DLMP aggregation with the dladm command, and the configuration is quite similar to that of a trunk aggregation. You can even transform a trunk aggregation into a DLMP aggregation. However, both methods have different capabilities and constraints:
- With a trunk aggregation, all cables have to terminate in the same switch; with DLMP you can connect each interface to a different switch.
- Trunk aggregation can spread the load across all member interfaces of an aggregation. DLMP cannot do this in the same way — with DLMP you only get HA when you have a single VNIC. However, when you configure multiple VNICs on an aggregation, DLMP will distribute those VNICs across the member interfaces.
- Because DLMP doesn’t support load spreading, you don’t configure a load-spreading policy for it.
- With trunks, the switch has to support trunks and must be configured to accept such aggregations (or you have to use LACP). A DLMP aggregation, by contrast, needs no support and no configuration on the switch.
How to Use It
DLMP is really easy to use. Compared to setting up a trunk aggregation, it just requires an additional -m dlmp:
root@solaris:/# dladm create-aggr -m dlmp -l net0 -l net1 aggr0
With show-link you will see that you have an additional data link — an aggregation consisting of net0 and net1, just as we configured a second ago:
root@solaris:/# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
aggr0 aggr 1500 up net0 net1
With show-aggr you can look up more details:
root@solaris:/# dladm show-aggr -x
LINK PORT SPEED DUPLEX STATE ADDRESS PORTSTATE
aggr0 -- 1000Mb full up 8:0:27:29:a2:89 --
net0 1000Mb full up 8:0:27:29:a2:89 attached
net2 1000Mb full up 8:0:27:ec:8a:7e attached
Now unplug one of the cables:
root@solaris:/# dladm show-aggr -x
LINK PORT SPEED DUPLEX STATE ADDRESS PORTSTATE
aggr0 -- 1000Mb full up 8:0:27:ec:8a:7e --
net0 0Mb unknown down 8:0:27:29:a2:89 standby
net2 1000Mb full up 8:0:27:ec:8a:7e attached
Neat.
Conclusion
One use case for this feature is solving a constraint introduced by using VRRP to make the integrated load-balancing feature highly available, as described in this blog entry. Alternatively, you can use it to give all the VNICs you use for zones a redundant connection to the network without having to work with IPMP. This also makes it much simpler to introduce things like the network resource manager on top of resource-controlled VNICs, compared with doing the same on IPMP. Such configurations get complex very quickly, and DLMP can reduce that complexity.
Further Reading
docs.oracle.com — Managing Oracle Solaris 11.1 Network Performance
