[Beowulf] dedupe filesystem

[Kilian CAVALOTTI]

On Wednesday 03 June 2009 14:55:52 Michael Di Domenico wrote:
> Do you find such a policy hard to enforce with researchers?  I don't
> have tiered storage today, but in the future i can see a need to have
> a storage pool with SATA and a storage pool with SAS or faster drives
> in it.  Some of the researchers where I am, work on data for months.
> Is this something better solved with pre/post-amble copies or through
> policies?

The best of both worlds would certainly be a central, fast storage filesystem, 
coupled with a hierarchical storage management system.

Oh wait, it might exist already... Well, at least it's in the works: Sun and 
CEA are working on implementing such an HSM for Lustre 2.0. See 
http://wiki.lustre.org/images/8/8b/AurelienDegremont.pdf for details.

Cheers,
-- 
Kilian