Separated ZIL on ramdisk.

Quicksearch

Disclaimer

The individual owning this blog works for Oracle in Germany. The opinions expressed here are his own, are not necessarily reviewed in advance by anyone but the individual author, and neither Oracle nor any other party necessarily agrees with them.

Navigation

< Spleen | Over 900 subscribers >

Separated ZIL on ramdisk.

Friday, October 10. 2008

I used ramdisks today in a live presentation of ZFS, L2ARC and sZIL (you can´t use files for sZIL and L2ARC testing, you need a device). One of the customers asked me if there is a real use case for this configuration. My answer was: Not really. But this question haunted me the whole day while sitting in the train.

I had some weird ideas like using an UPS backed server to act as a sZIL device by provding access to a ramdisk via Fibre Channel (by using the FC target in COMSTAR) , but i had no idea if this would be really clever idea. Ramdisk based sZIL? Okay, i know there are now several admins with a cardiac arrest out there. The problem of a ramdisk based sZIL is the volatile nature of the ramdisk. When you loose the power on your systems, you have still a consistent pool but you loose the transactions stored in the sZIL.This is a less than desirable effect. Between you and loosing transactions is just the power switch

There was a missing piece...

The good thing about a large community: Sometimes someone had already similar ideas long ago and had some time to do some experiments. So i found a really neat idea on the zfs-discuss mailing list with the important extra thought compared to my "dozing in the train"-idea.

The idea of Chris Greer has this nice additional twist making the concept of using a ramdisk sZIL more sensible. He writes in "An slog experiment (my NAS can beat up your NAS)":

So I tried this experiment this week...
On each host (OpenSolaris 2008.05), I created an 8GB ramdisk with ramdiskadm. I shared this ramdisk on each host via the iscsi target and initiator over a 1GB crossconnect cable (jumbo frames enabled). I added these as mirrored slog devices in a zpool.

This is really a neat trick. You can mirror a sZIL device so you could distribute the ramdisks over several systems. When one of the system fails, you have still the other device with the same data. This configuration had an impressive effect on the performance

The big thing here is I ended up getting a MASSIVE boost in performance even with the overhead of the 1GB link, and iSCSI. The iorate test I was using went from 3073 IOPS on 90% sequential writes to 23953 IOPS with the RAM slog added. The service time was also significantly better than the physical disk.

I think you can drive this concept even farther by substituting the iSCSI over TCP over Gigabit with iSER (iSCSI over RDMA over Infiniband) and using separate small x86 systems for this task, each backed by a small UPS keeping the system up until an dd from the ramdisk to a hard disk has completed.

Chris, sounds like a really clever idea ...

Posted by Joerg Moellenkamp in English, Solaris at 20:28 | Comments (4) | Trackbacks (2)

Defined tags for this entry: l2arc, solaris, szil, zfs

Related entries by tags:

View as PDF: This entry | This month | Full blog

Trackbacks

Trackback specific URI for this entry

RAMdisks for the L2ARC of ZFS
I thought a little bit about the idea of transforming server into solid state disks. The idea in the mail of Chris Greer on zfs-discuss was to use mirrored iSCSI shared ramdisks as a storage for the seperated ZILs. But i think you could use the concept as

Weblog: c0t0d0s0.org
Tracked: Oct 13, 18:58

PingBack

Weblog: zfs.se
Tracked: Nov 07, 17:54

Comments

Display comments as (Linear | Threaded)

COMSTAR with SAS Target sounds very cool for this setup...But how long is data kept in the sZIL device before it is flushed to disk? Isnt it enough to keep the data in the write cache of the sZIL computer in case your zpool node crashes? So this way you could use a disk-backed sZIL device that survives everything except rebooting both/all sZIL nodes in parallel.

#1 Olli on 2008-10-11 14:06

LOL. Mirrored Ramdisk ZILs don't work in production environment because if the servers are on the same power grid, you are screwed if the entire power grid went offline.

Now mirrored ramdisks backed by battery and flash(Acard) might work. Although the Acard ram drive cannot be hot swapped so in production, if one of the ram drives failed, you have to offline the system anyways, which defeats the purpose.

Some one please make a battery backed ramdrive that's backed by integrated flash and make it hotswappable 2.5inch. That device would sell like hotcakes. For now mirroring SLC drives would do. (Look out for Samsung's enterprise drives)

#2 TS on 2008-12-09 20:21

That´s easy ... small rackable USV at the server ... just for long enough to dump the ramdisk to a real disk ...

#2.1 Joerg+M. (Homepage) on 2008-12-09 20:30

ACADR or any other RAM-SSD are limited to 1.2-1.3 GB/s RAID cards throughput limit, Ramdisk on the server with 20GB Infiniband (40GB might be used later with Nehalem Xeon platform) will work at 1.6-1.8GB/s. Same goes for IOPS, as Infiniband connection is less latent than RAID one.

#2.2 Stanislav D. (Homepage) on 2009-02-16 20:18

Add Comment

Name
Email
Homepage
In reply to
Comment	E-Mail addresses will not be displayed and will only be used for E-Mail notifications. To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly. Enter the string from the spam-prevention image above: Standard emoticons like :-) and ;-) are converted to images. Enclosing asterisks marks text as bold (word), underscore are made via _word_.
	Remember Information? Subscribe to this entry