[Beowulf] OT: recoverable optical media archive format?

[Lux, Jim (337C)]

> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On Behalf Of David N. Lombard
> Sent: Thursday, June 10, 2010 3:12 PM
> To: David Mathog
> Cc: beowulf at beowulf.org
> Subject: Re: [Beowulf] OT: recoverable optical media archive format?
> 
> 
> The website is more interested in corrupted block media, with the assumption
> said corruption manifests as a cluster of invalid blocks from the file.  You've
> got a different type of corruption.
> 



In the communications field one uses interleaving as a way to turn burst errors into isolated errors.  One then uses a short length coding scheme to detect and correct the isolated errors.

On systems where the errors are sporadic (communication links with additive white Gaussian noise, or RAM that gets sporadic single bit flips), then interleaving doesn't buy you anything, and you can use short codes like Hamming or Reed-Solomon/BCH.

On systems where errors are transient (read errors from flash.. read the same location again a second time and it's ok) or where  you can ask for a repeat, then block oriented "go back N" codes work well.   Unless you have a timing determinism requirement or the retry interval is very long (8 hour light time to Pluto), and then, some sort of redundant block scheme (send every block three times in a row) gets used.

A good practical example is the coding used on CDs... the error correcting code is a Reed Solomon, which does real well at isolated errors, but not great at burst errors.  So they use a R-S code with a block interleave scheme in front of it and then another R-S code, because the error statistics from CD drives show burst errors.


So it looks like you are looking for the inverse.. something that turns distributed errors into clumps?