c0t0d0s0.org

I mentioned the update of the 7410 hardware in a blog entry yesterday afternoon. Brendan wrote an interesting blog entry about the performance impact with "7410 hardware update, and analyzing the HyperTransport". For cached workloads (to get the disks out of the equation), the increase of performance is really impressive:

Workload	7410 (Barcelona)	7410 (Istanbul)	Improvement
NFSv3 streaming cached read	2.15 Gbytes/sec	2.68 Gbytes/sec	25%
NFSv3 8k read ops/sec	127,143	223,969	75%

What many people forget when talking about Istanbul, is the fact that it didn't get just two additional cores. You get HT3 and HT Assist (a snoop filter to reduce bus contention by processors searching their data in other processors caches) with it. Brendan summarized his findings with:

For this 7410 upgrade, the extra CPU cores help - but it's more about the upgrade to the HyperTransport. HT3 provides 3x the CPU interconnect bandwidth, and dramatically improves the delivered performance of the 7410: from 25% to 75%.

The Sun Storage 7410 got a little update. You can get it with up to four six-core CPUs, thus up to 24 cores in total.

The Exadata V2 announcement contained a hint to a new product: The Aura card. The official name of this card is Sun Flash Accelerator F20 PCIe Card. Finally this product found it's way to the Sun Website. The specs are really nice. The 96 GB variant delivers 100.000 read IOPS and 87.000 write IOPS at 4k. The card contains a super-cap to ensure that there is power to write down the caches to the flashes. 1092 MB/s read throughput, 494 MB/s for writes. In addition to this it offers SAS connectivity for 8 drives, thus you loose nothing when you use this card instead of a normal SAS controller. You just have additional flash disks.

In the article about the new software version for the S7000 series i mentioned a feature called Shadow Migration. Eric Schrock wrote an interesting article about the inner workings of this feature in his blog: What is Shadow Migration?. I wonder if Solaris Engineering could use this work for a CacheFS follow-on .... at the end this would be not much more than a perpetual shadow migration plus the cache logic (LRU-stuff and consistency checking).

There is a new version of the software for the Sun Storage 7000 series. You can download it at the usual location. This software version provides a lot of new features:

• Infiniband Support via IPoIB
• Improved iSCSI
• Preview of a NFS/RDMA
• User quotas and space accounting
• In wide-stripe configurations the triple parity RAIDZ3 substitutes dual parity RAIDZ2
• Shadow migration. You can use this feature to migrate data from other systems with little downtime, as it is done when the data is accessed by the users
• vastly faster cluster-failover times

You will find a complete list of changes in the release notes at wikis.sun.com

I write my opinion about the new Exadata machine a little bit later, but i found another interestin thing about TheRegisters Timothy Prickett Morgan while scanning news about the Exadata stuff. He writes in Oracle, Sun speed-launch Exadata V2:

If Oracle is trying to convince Sun customers that it is committed to the Sparc platform, perhaps it is not trying hard enough.

A few moments later i've read in Oracle, Sun Show Off Super-Fast, Flash-based OLTP Server:

Bernie Spang, IBM's Director of Data Management Product Strategy ... It was "interesting that this is an x86-based system after they [Oracle] ran full-page ads attempting to calm fears regarding commitment to Sun's SPARC," Spang said.

Draw your own conclusions. At least I don't try to make the impression that i'm not partisan ....

So ... 16.000 pageviews later, i've read through several reactions to my article "Some perspective to this DIY storage server mentioned at Storagemojo". I wrote that article after reading Robin Harris' article about a DIY high density storage server. Thanks to Google Translate even those in russian language (i've realized that my french is still good enough to read through texts, albeit i have to admit that reading a text in french written by me would be an even more horrible experience than a text in english). Just a warning at the beginning. This text is 32 hard disk marketing KBytes long. 31 of the real ones

Introduction

Okay, the following article isn't really about this Backblaze device, it's more about some misunderstandings regarding storage. For every suggestion i will make in this article, i see a reason why Backblaze didn't go this way. But everyone who just looks at the price and think of Sun Hardware as gold-plated, should rethink. Because for everything Sun (or other storage vendors) did at their devices, there are several good reasons as well.

The only think i've really hated about this article at Backblaze was this figure about the costs. Many people just reposted this figure but not the statement that clearly said, that this isn't a general purpose device. Without their software this storage pod would be just an horrible example to explain how you shouldn't design storage. With their software it's may be an adequate solution for their special needs and workload, and just for their special needs and their workload. No wonder, as the just developed this software for this use case.

Latency matters. Even in small quantities and thinking about storage networking just in IOPS without thinking about latency is like thinking of disks just in size but not in IOPS. The impact of latency isn't just a theoretical one, as some observers suggest.

This latency thing wasn't such a problem in the past. You've only talked about it, when cabling over longer distances (100km for example) came into the focus. But with the increased popularity of SSD this starts to be a problem. No ... it's already a problem. I just want to guide you through a thought game to show you this challenge.

I've received yesterday some mails/tweets with hints to a "Thumper for poor" DIY chassis. Those mails asked me for an opinion towards this piece of hardware and if it's a competition to our X4500/X4540. Those questions arised after Robin Harris wrote his article "Cloud storage for $100 a terabyte", which referred to the company Backblaze, which constructed a storage server on its own and described it on their blog in the article "Petabytes on a budget: How to build cheap cloud storage". Sorry, that this article took so long and there may be a higher rate of typos, as my sinusitis came back with a vengeance ... right in the second week of my vacation. But now this rather long article is ready

Sun talked about a flash array at the ISC in Hamburg. Now it got obvious, that something is imminent in regard of this system, as soon as patch 141484-01 was released a few days ago. Yesterday the news broke about some documents at docs.sun.com about an interesting upcoming device. Now Storagemojo wrote about it

Kudos to the Sun engineers who have been driving flash forward faster than almost anyone else in the industry.

as well as Octave Orgeron in his blog who stated

This will be Sun's first step into selling products that could very well change the storage landscape.

Perhaps those was fueled by this teaser blog entry yesterday, but there was an other article about it at blogs.sun.com. Looks i'm not the only one traversing docs.sun.com at a regular schedule.

This device is really something really interesting, but i won't write more about it until it's officially announced. Oh ... just one teaser question: What's the really slow and performance-annihilating part at swapping virtual memory from/to disk?

Sometimes you stumble over interesting upcoming products already documented in docs.sun.com: Click on Storage, then at the "Browse Product Documentation" at Hardware (if it's not already opened), then click on Disk storage. Now open the Solid State Storage part of the tree. And hopefully you will find the documentation of a really interesting device.

I got aware of a new rumour in regard of Oracle and Sun: This rumour is based on a CNN article, which speculated that Oracle will sell of the hardware business of Sun to HP. I just dismissed it as a large heap speculation. They have to speculate as the media doesn't have any insight knowledge and i didn't wanted to write about it. Okay, but now there is an additional twist in the story. I joked about it a few days ago that this will provoke exactly this reaction from the tin foil hat department.

The Sun+Oracle is faster web page was available under the URL http://www.oracle.com/features/hp/sunoraclefaster.html, later it moved to http://www.oracle.com/features/sunoraclefaster.html. I assume, you know what people thought now? Yes ... the /hp/ part in the URL is a sure sign that the rumour at CNN is true. Really, some people think that and i've heard, there are some people at competitors, who fuel this thoughts.

Folks, could you please do at least a little bit of fact checking? Let's look at the Oracle website.
http://www.oracle.com/features/hp/middleware-strategy.html ... ehh ... no HP here, http://www.oracle.com/features/hp/oracle-database-11g.html ... sorry ... it's about 11g, not something in relation to HP. Oh, http://www.oracle.com/features/hp/innovation-on-demand-crm.html ... nope ... no luck ... no HP in there. Okay, people with the nice silver hats ... it may be possible that /hp/ may stand for something like home page, high performance, hillarious projections or even horse poo. Okay, home page looks like the most probable explanation as those stories were homepage features. And vice versa: http://www.oracle.com/openworld/2008/hp.html, that's an article about HP at the Openworld, no /hp/ in this URL. Or this piece about the Exadata Storage Server at http://www.oracle.com/profit/features/exadata.html. Heck ... no /hp/ in this url, too.

Okay, okay ... i know what comes next: Oracle felt catched in their intentions about Sun HW business and renamed the page out of this reason. Folks, entia non sunt multiplicanda praeter necessitatem. Perhaps there is the remote possibility there was someone at Oracle who thought that putting Sun, Oracle an the to letters h and p in the same line isn't such an intelligent move, when you want to brag about the performance of Sun equipment. And perhaps someone at Oracle thought, it isn't a good idea to have this URL, when you don't want to fuel any rumours from outsiders or FUD from competitors, especially in the light of such blatant speculation pieces like the CNN article, which was published a day earlier.

And to close this article: There are dozens of reasons why the CNN article is just unfounded speculation, but i just want to hint to a single line. Oracle wrote in the announcement for the TPC-C result publication on October 14th:" Check back on October 14 as we demonstrate Oracle's commitment to Sun hardware and Sun SPARC.". Enough said.

I really like Robin Harris' Storage Mojo blog, but in this case, but i'm thinking in his article Why did Apple drop ZFS he is wrong, when he thinks that the CDDL may posed a problem: Mac OS X uses Dtrace for a while now and it's licensed under the CDDL as well. So this can't be the reason. I assume some technical problems, but at Apple this includes "Steven didn't like the graphical user interface". Considering that this was a release concentrated at the inner workings of MacOS X, they didn't wanted to introduce a completely new feature to reduce potential instabilities because of newly introduced code. I don't even think a GPLed ZFS would help, given that a GPLed ZFS would introduce the same kind of problems for MacOS X than a GPLed Linux has with the CDDLed ZFS.

While commenting to a tweet of @alecmuffet a question arised in my head: Why do many people talk about SAN boot and why do people seldomly ask for NFS boot?

Asides from my opinion that systems should be able to boot without help from the outside (debugging is much easier, when you have an OS on your metal), SAN boot looks to me like combining the worst of all worlds. You depend on centralized infrastructure, you have to put HBAs (at least two) in every server, you have to provide an additional fabric, encryption is still an unsolved problem.

On the other side you have NFS, deduplication and cloning is a non-problem when you configure your boot environment in a clever way, encryption is available by the means of IPsec. Caching to get load from the network is possible by CacheFS for example. Or think about the combination of NFS boot with the snapshot/clone function of ZFS to clone you boot environments?

So why is everybody looking in the direction of SAN boot instead of using NFS boot, when they look at centralized boot storage? The issue of needing an additional fabric is known, so solutions are developed, but instead of simple going in the direction of NFS the industry is running into the direction of blocks over IP or blocks over Ethernet. Strange world ...

Perhaps that's interesting for people needing a highly secure tape storage. As reported by eWeek the StorageTek T10000B tape drive got the FIPS 140-2 certification:

Nonetheless, Sun revealed Aug. 19 that it has become the first enterprise tape drive maker to be granted a prestigious federal security qualification: the FIPS 140-2 Certification at Security Level 2 for its Sun StorageTek T10000B tape drive.

The T10000B drive has the integrated capability to encrypt the data before writing it to the tape. Thus it contains components to do this encryption. The FIPS certification states that a hardware device complies to the standards set by the FIPS 140-2 document, which is headlined "Security Requirements for Cryptographic Modules". A level 2 FIPS certification means (copied from the Wikipedia article):

Security Level 2 improves upon the physical security mechanisms of a Security Level 1 cryptographic module by requiring features that show evidence of tampering, including tamper-evident coatings or seals that must be broken to attain physical access to the plaintext cryptographic keys and critical security parameters (CSPs) within the module, or pick-resistant locks on covers or doors to protect against unauthorized physical access.

Thus you can't get the key without leaving traces. BTW: I'm sure the messages in "Mission: Impossible" are FIPS140-2 Level 5 certified ... "This tape will self-destruct in five seconds"

The T10000B is the first tape-drive with this level of certification. If you are interested in this matter, the certification of the T10000B is available the nist.gov website. The FIPS140-12 document itself is available for download at NIST, too.