Quicksearch
Navigation
Events
Trusted Ad
Blower Door Test
Luftdichtigkeitsmessung für Gebäude Raum Hamburg ab 160 € excl. MwSt
www.m-tectum.de
Luftdichtigkeitsmessung für Gebäude Raum Hamburg ab 160 € excl. MwSt
www.m-tectum.de
Kategorien
About this blog- Apple
- iPhone
- Aviation
- Bahn
- Begriffe des Lebens
- Blogosphere
- Braindump
- Business Travel
- CANBOX
- CeBIT
- CEC
- Computing
- del.icio.us
- Fundsache
- General
- i hate sundays
- Kochen
- Mindfuck
- Movies
- Music
- On Benchmarketing
- Persoenliches Pech
- Photographie
- policy of ...
- Privacy
- Recommended Blogs
- Security
- Solaris
- Strange Days
- Sun
- suncec2007
- suncec2008
- Technology
- The IT Business
- Toolbox
- Travel
- Work
Presentation: End of RAID5
Friday, February 22. 2008
I often use a thoughtgame when i talk about ZFS: Can we afford to use large harddisks without stronger mechanisms to ensure data validity? The best examples for this problem came from Robin Harris and his Storagemojo.com blog with the article about the reliability of hard disks. I mostly use this example as an anecdote but i transformed it into a draft presentation in the last few evenings:
In my opinion the problem is much worser than described in Robins blog entry: At first you have the time bomb effect of transient read errors described in the presentation. Furthermore better error correction doesn´t solve the problem completly. There are many components between the heads of the harddisks and the CPU. You need checksums for the data from the disk to the CPU. Everything else isn´t a complete solution of storing data on rotating rust while ensuring data validity.
In my opinion the problem is much worser than described in Robins blog entry: At first you have the time bomb effect of transient read errors described in the presentation. Furthermore better error correction doesn´t solve the problem completly. There are many components between the heads of the harddisks and the CPU. You need checksums for the data from the disk to the CPU. Everything else isn´t a complete solution of storing data on rotating rust while ensuring data validity.
Comments
Display comments as
(Linear | Threaded)
As I am doing a lot of proof-reading these days, please allow to comment on three things:
1st: Minor error: English on page 9 (fron -> from; Statisticially -> Statistically) and maybe a comma after "Statistically" but I am not sure with the latter.
2nd: logical error
You are projecting the error rates of an 120GB disk for a hypothetical 2 TB disk. Ok, lets accept that (at first glance).
However your conclusion with 1 error bit after the RAID5 reconstruction is wrong. Yes, there is one error every 12 TB, but only if you read 12 TB from one disk. Therefore the RAID setup in your example is still @100% integrity, because you only red 2 TB per disk.
Therefore, the RAID setup in your example would be safe for ~6 RAID reconstructions, if we do not count reads during normal operation.
3rd: Biggest "flaw" however, is the initial assumption with the 10E14 error rate. That is the error rate of a 120 GB desktop part. I would use server parts in a raid setup, e.g. the Seagate Cheetah® NS. The data sheet is available here:
http://www.seagate.com/docs/pdf/datasheet/disc/ds_cheetah_ns.pdf
There you can see that the error rate for that server disk is 10E16. If we would calculate your example with that number, you would end up with an invalid bit every 1.136,9 TiB. In case for a 400 or 300 GB server harddisk.
In the end the only thing which is proofed, is Winston Churchill's famous quote on statistics
cheers
Alex
1st: Minor error: English on page 9 (fron -> from; Statisticially -> Statistically) and maybe a comma after "Statistically" but I am not sure with the latter.
2nd: logical error
You are projecting the error rates of an 120GB disk for a hypothetical 2 TB disk. Ok, lets accept that (at first glance).
However your conclusion with 1 error bit after the RAID5 reconstruction is wrong. Yes, there is one error every 12 TB, but only if you read 12 TB from one disk. Therefore the RAID setup in your example is still @100% integrity, because you only red 2 TB per disk.
Therefore, the RAID setup in your example would be safe for ~6 RAID reconstructions, if we do not count reads during normal operation.
3rd: Biggest "flaw" however, is the initial assumption with the 10E14 error rate. That is the error rate of a 120 GB desktop part. I would use server parts in a raid setup, e.g. the Seagate Cheetah® NS. The data sheet is available here:
http://www.seagate.com/docs/pdf/datasheet/disc/ds_cheetah_ns.pdf
There you can see that the error rate for that server disk is 10E16. If we would calculate your example with that number, you would end up with an invalid bit every 1.136,9 TiB. In case for a 400 or 300 GB server harddisk.
In the end the only thing which is proofed, is Winston Churchill's famous quote on statistics
cheers
Alex
1. Will change that
2. The number doesn´t say "You read one corrupt sector every 12 TB". It says the probability of reading an corrupt byte 12 TB is 1.
3. You are right, i wouldn´t use desktop drives too, but i´ve heard "You harddrives are so expensive" too often. There are resons for the price of enterprise class FC drives. By the way: The error rate of enterprise class sata is 10E15 not 10E16.
2. The number doesn´t say "You read one corrupt sector every 12 TB". It says the probability of reading an corrupt byte 12 TB is 1.
3. You are right, i wouldn´t use desktop drives too, but i´ve heard "You harddrives are so expensive" too often. There are resons for the price of enterprise class FC drives. By the way: The error rate of enterprise class sata is 10E15 not 10E16.
2: Well, lets say that the "probability" for an error is far too high for a production environment
I think that is the point where your ZFS argumentation should start. There could be errors, i.e. your data is not safe at all. The "funny" thing with statistics in that case is also that you do not know when that error might occur. Even the very first sector read could be corrupted, or the last one or one in between the 10E1X Therefore it does not even matter if it is 10TB or 1000TB ...
In any case it is unacceptable for high important data. A file system with checksums is a much better solution.
3: Hmmm, 10E15 ? Have you red the above link about the Cheetah disk ? There it says 10E16. However I had a look into the other Seagate data sheets on their webpage and there are also 10E15 enterprise drives. All in all there seems to be three difference error-rate levels:
desktop SATA: 10E14
"standard" enterprise SATA/SAS: 10E15
"high end" enterprise SATA/SAS: 10E16 (Cheetah and Savvio series)
I have not checked other companies, maybe Seagate is the only company with 10E16. Seems like they are quite proud of it, they also mention an "exclusive" 10bit ECC implementation for the Cheetah series.
4. Disk pricing:
I guess your comment is due to the fact that OEMs like Sun, IBM, ... often buy harddisks, RAM modules, etc., and sell that piece of hardware later under their own label, with own warranty, services and so on. The resulting price however is then mostly (far?) more expensive than the market prices of the original e.g. Seagate harddisk, Micron RAMs, etc.
Otherwise your argument would mean that people want to buy a Porsche (i.e. high-end server with best reliability) but want to pay a Volkswagen's price, only. Sounds too crazy to me.
However ... business people with their attitude to (over)optimize the ROI .. hmmm could be true ^^
cheers
Alex
I think that is the point where your ZFS argumentation should start. There could be errors, i.e. your data is not safe at all. The "funny" thing with statistics in that case is also that you do not know when that error might occur. Even the very first sector read could be corrupted, or the last one or one in between the 10E1X
Therefore it does not even matter if it is 10TB or 1000TB ...In any case it is unacceptable for high important data. A file system with checksums is a much better solution
. 3: Hmmm, 10E15 ? Have you red the above link about the Cheetah disk ? There it says 10E16. However I had a look into the other Seagate data sheets on their webpage and there are also 10E15 enterprise drives. All in all there seems to be three difference error-rate levels:
desktop SATA: 10E14
"standard" enterprise SATA/SAS: 10E15
"high end" enterprise SATA/SAS: 10E16 (Cheetah and Savvio series)
I have not checked other companies, maybe Seagate is the only company with 10E16. Seems like they are quite proud of it, they also mention an "exclusive" 10bit ECC implementation for the Cheetah series.
4. Disk pricing:
I guess your comment is due to the fact that OEMs like Sun, IBM, ... often buy harddisks, RAM modules, etc., and sell that piece of hardware later under their own label, with own warranty, services and so on. The resulting price however is then mostly (far?) more expensive than the market prices of the original e.g. Seagate harddisk, Micron RAMs, etc.
Otherwise your argument would mean that people want to buy a Porsche (i.e. high-end server with best reliability) but want to pay a Volkswagen's price, only. Sounds too crazy to me.
However ... business people with their attitude to (over)optimize the ROI .. hmmm could be true ^^
cheers
Alex
The LKSF book
The book with the consolidated Less known Solaris Tutorials is available for download here
Web 2.0
Contact
Networking
open.bc
LinkedIn
Facebook
My photos
- mail: joerg@c0t0d0s0.org
- icq: 144000169
- Me, myself and I
- openBC
- My del.icio.us
- c0t0d0s0@twitter
- c0t0d0s0@last.fm
- wiki@c0t0d0s0
- Amazon Wishlist
Networking
open.bc
My photos
Syndication
Comments
about Hardware teaser: SPARC Enterprise T3120
Thu, 20.11.2008 20:21
Tenaja will be an interesting product that will be on the ma rket in a few months, just sta y tuned
Thu, 20.11.2008 20:21
Tenaja will be an interesting product that will be on the ma rket in a few months, just sta y tuned
about My thoughts about the future of Sun ...
Thu, 20.11.2008 18:13
Wahrscheinlich hat man bei den anstehenden Kosten auf Office 2007 mal darauf geguckt, ob d as wirklich so wichtig u [...]
Thu, 20.11.2008 18:13
Wahrscheinlich hat man bei den anstehenden Kosten auf Office 2007 mal darauf geguckt, ob d as wirklich so wichtig u [...]
about My thoughts about the future of Sun ...
Thu, 20.11.2008 16:23
Das letzte Mal wie OO "hipp" w ar, hieß es, dass es an den vi elen, vielen firmenspezifische n Excel Plugins scheiter [...]
Thu, 20.11.2008 16:23
Das letzte Mal wie OO "hipp" w ar, hieß es, dass es an den vi elen, vielen firmenspezifische n Excel Plugins scheiter [...]
about My thoughts about the future of Sun ...
Thu, 20.11.2008 13:18
Fishworks is the way to get (O pen-)Solaris into companies wh ere Solaris is not strategic ( e.g. also Windows-only s [...]
Thu, 20.11.2008 13:18
Fishworks is the way to get (O pen-)Solaris into companies wh ere Solaris is not strategic ( e.g. also Windows-only s [...]
Getaggte Artikel
AMD Apple avs Bahn Blogging Blogosphere braindump Business Travel CeBIT cec cec2006 CMT del.icio.us deutsch dtrace fliegen Fundsache General Hamburg IBM i hate sundays Intel iscsi jumpstart Links Linux lksf Mindfuck Movies Music Musik Niagara Opensolaris Opteron Photographie policy of ... Politik Security Solaris storage Sun suncec2007 sunw t1 The IT Business Ultrasparc ultrasparc t1 Wirtschaft Work ZFS
Blog Administration
Buttons
A few days ago, i posted the first draft of a presentation about the death of RAID5 and about the advantages of ZFS to solve the problems . Infoworld´s Mario Apicella writes in The death knell for RAID? about this topic with a different viewpoint, but he
Tracked: Mar 02, 16:39