c0t0d0s0.org

While sitting in a workshop meeting, my thoughts went away from the meeting and circled around the Turbo Boost feature of the Nehalem processor. Too make it clear at the beginning: Turbo Boost is a good feature. It helps to use the existent potential of the CPU instead of the expected potential. Overclockers have shown in the past, das most CPU have more performance potential as the print on the heat spreader says. But while I think this is a good idea, this will open some interesting benchmarketing tricks.

Brendan Gregg did an interesting benchmark with our new storage systems to show what an 7410 single-head can deliver: A quarter million NFS IOPS. This number was generated with a benchmarking configuration - 1 byte blocksize. This isn´t really realistic - it´s just for benchmarking. But he delivers a more meaningful result in his text, too:

I've reached over 145,000 NFSv3 read ops/sec - and this is not the maximum the 7410 can do (I'll need to use a second 10 GigE port to take this further). The latency does increase as more threads queue up, here it is plotted as a heat map with latency on the y-axis (the darker the pixel - the more I/Os were at that latency for that second.) At our peak (which has been selected by the vertical line), most of the I/Os were faster than 55 us (0.055 milliseconds) - which can be seen in the numbers in the list on the left.

Phil ran a memory benchmark and summarized his results in Niagara 2 memory throughput according to libMicro:

[...] seeing 240ns latency, which equates to a throughput of 267 million memory reads per second (i.e. 64 / 0.240e-6). Just to set this in context, here are some data for a quad socket Tigerton system running at 2.93GHz...
[...] his shows a peak throughput of about 86 million memory reads per second (i.e. 16 / 0.186e-6), making the single chip UltraSPARC T2 processor's throughput 3x that of its quad chip rival.

It´s a micro benchmark, but it´s a part of the puzzle why an UltraSPARC T2 is able to outperform a processor with almost 2.5 times more clock cycles per seond.

I thought a little bit about the TPC-H benchmark published by IBM yesterday. There is something in it, that leaves some question. The p570 is a 8 socket system. You can build it by stacking 4 two-socket boxes with some special cabling. That´s one of the points behind buying a p570. Nevertheless, the configuration of the benchmark didn´t worked that way. They had 32 of this p570 building bloacks and used this boxes as 32 distinct systems making the p570 buildingblocks to an expensive volume server. By using the capability to build bigger systems ot of those building blocks you would have 8 instead of 32 systems to manage.

The questions: What´s the reason of using such a configuration? Is there a scalability problem within the p570 giving you less return for the socket in bigger configurations ? What´s the impact to loads with higher dependencies between the partitions? I would like to see the performance numbers for a system with 32 p570 buildingblocks in 8 systems with an otherwise equal configuration.

Wow ... the Sun T5120 is the first single socket system to break the 10.000 SAPS barrier (you will find the certifcate here) ...

Peerapong Kunasirirat points to some really interesting benchmark documents regarding the comparision between ZFS and ext3/NTFS/vxfs. I find one paragraph out from the ext3/ZFS comparision quite informative:

For the conditions tested, Solaris ZFS can outperform the Red Hat Enterprise Linux ext3 file system for many workloads, especially Postmark and relational database indexed queries. In other cases, Solaris ZFS exhibits comparable performance but does not require the performance or data integrity trade-offs that are inherent in the ext3 file systems when running in ordered and writeback modes.

Yet another benchmark of ZFS: This time at Robert´s blog.The benchmark was done with the focus of comparing hardware and software raid configurations. He sees the replacement of failed disk with ZFS as an little weakness, but this is stuff engineers already work on. But at the end, the he comes to the conclusion:

As it looks like in many workloads HW RAID will in real life give you less performance for higher price... but you get all the other features, right? Well, you get clones, snapshots... but you get them buit-in with ZFS and for most workloads ZFS clones and snapshots not only will give much better performance but won't need dedicated disks. Then management is much more easier with ZFS than with arrays - especially when you think about different software to manage arrays from different vendors. With ZFS it's all the same - just give it disks...

It´s one of the long lasting myths, that RAID needs hardware to be fast. It´s like "Disable autoneg" and "1 MHz per MBit". Those rules were true in the past but obsoleted by the time. But the RAID myth seems to be more persistant than all other before. Whenever i talk about ZFS RAID, the people stare at me and and i see the though "Oh, no, the guy talks about slow SW RAID". Okay, when you don´t believe me perhaps you believe a user our products: Robert Milkowski gives some numbers on RAID performance here and here.

Colm made an additional benchmark. The T2000 seems to be faster on Linux than on Solaris, but I assume that he hadn´t switched on the write cache of the discs. This can make quite a difference. On Linux the write caches are activated by default.

This happens when you use a system for a tailor-made task:

Bottom line, the T2000 was able to handle over 3 times the number of transactions per-second and about 60% more concurrent downloads than the current ftp.heanet.ie machine can (a dual Itanium with 32Gb of memory) running identical software.

The full review is available here.