[Beowulf] Storage - the end of RAID?

[Lux, Jim (337C)]

> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On Behalf Of Joe Landman
> Sent: Friday, October 29, 2010 1:10 PM
> To: beowulf at beowulf.org
> Subject: Re: [Beowulf] Storage - the end of RAID?
> 
> RAID IS NOT A BACKUP (can't say how many times I've had to say this to
> customers).  It can (and does) occasionally fail.  The only *guaranteed*
> way to prevent the failure from increasing entropy significantly in the
> universe is to have a recent copy of all the relevant data.
> 
> Which is RAID1 all over again.
> 

Could one not design a coding strategy that uses a bit more redundancy than the (8,3) Huffman code and that essentially doesn't need to be rebuilt.. that is, say you have 12 drives to store 8 drives worth of data, and for ease of talking, one bit/byte is written across the array.

A drive fails, and you can still read the data ok from the remaining 11 (and, in fact, tolerate another failure). You put in a new drive, which contains all "wrong" bits (actually, half the bits are wrong and half are right, but you don't know which are which)...

You read from the full array, and the bits that are wrong on the new array just get corrected during the read in the usual way.  You write to the array, and all 12 bits get written.  So gradually, the new drive gets filled with "correct" bits.  As long as you don't get another TWO failures before all the bits are ok, you're in good shape.

(yeah, you could do writeback on error to fill in erroneous bits in the background, etc. but I assume that's not an option because of performance).