[Beowulf] HD undetectable errors

[Reuti]

Am 25.08.2009 um 22:37 schrieb Nifty Tom Mitchell:

> <snip>
> However, It does make sense to me to generate and maintain site  
> specific meta
> data for all valuable data files to include both detection (yes tamper
> detection too) and recovery codes.

Sounds like using local par or par2 files along with their hash  
information about the original files. Maybe this could be implemented  
as FUSE filesystem for easy handling (which will automatically split  
the files, create the hashes and any number of par files you like).

-- Reuti


> I would extend this to all data with
> the hope that any problems might be seen first on inconsequential  
> files.
> Tripwire might be a good model for starting out on this.
>
> I should note that the three 'big' error rate problems I have  
> worked on
> in the past 25 years had their root cause in an issue not understood
> or considered at design time so empirical data from the customer was
> critical.  Data sheets and design document conclusions just missed  
> the issue.
> These experiences taught me to be cautious with storage statistics.
>
> Looming in the dark clouds is a need for owning your own data  
> integrity.
> It seems obvious to me in the growing business of cloud computing  
> and cloud storage
> that you need to "trust but verify" the integrity of your data.    
> My thought
> on this is that external integrity methods are critical in the future.
>
> And do remember that "parity is for farmers."
>
>
>
> -- 
> 	T o m  M i t c h e l l
> 	Found me a new hat, now what?
> a
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf