There is an interesting discussion in a Marcelo Leals blog article “Do you need L2ARC redundancy? I do…” at the moment: Does Solaris need a mirrored L2ARC in Solaris? I’m divided about that or to be exact, my thoughts are “Yes, would be nice, but …”
- You should keep in mind, that the system works without problems when a L2ARC device fails, you just lose cache
- At the moment the L2ARC is cold after each boot. In my opinion the loss of the warm-ness of the cache is more at risk due to other failure modes like power failure, planned and unplanned reboots or the fatal failure of other components than by the failure of an SSD without moving parts. We need L2ARC boot persistency before we need mirroring.
- The other thought is more difficult: Let's assume you have a multiterabyte database and you have two 64 GB SSD. What delivers better performance? 128 GB of unmirrored L2ARC or 64 GB of mirrored L2ARC? Of course, if your working set is just 64 GB than the unmirrored second disk would give you no advantage. But what happens if the working set is larger? The L2ARC is used round robin, so the mixture of data should be equally warm on all devices. So it's not that way, that you lose 100% of warm data, just 50% with two disks when one disk fails. Using 6 disks would end in losing 1/6 of the hot cache areas.
The interesting (and very workload depending question) is: What impacts your system performance more? Having less cache, but not the problem of loosing warm cache areas due to SSD failure or having more cache, but the slight chance that 50% of the cache may fail and may be cold afterwards.
What do you think about it?