[Beowulf] dedupe filesystem

[John Hearns]

2009/6/5 Mark Hahn <hahn at mcmaster.ca>:
>
> I'm not sure - is there some clear indication that one level of storage is
> not good enough?

That is well worthy of a debate.

As the list knows, I am a fan of HSMs - for the very good reason of
having good experience with them.

There are still arguments made that 'front line tier = fast
SCSI/fibrechannel disk' 'second line and lwoer tier = SATA' and the
sales
types say SATA is slower and less reliable.

Mark, you make the very good point that the world is changing (or
indeed has changed) and you should be looking at an infinitely
expandable disk based setup - just add more disks into the slot, more
JBODs, whatever.
Actually, as a complete aside here I have been lookign at Virtual Tape
Libraries. One of the Spectralogic models actually eats SATA drives
just liek they are tapes -

I'm now going to counter your argument - let's say we have an
expensive parallel filestore such as Panasas.  Or maybe Lustre.
So your researchers work on a new project, and need new storage. But
they have old projects lying around.
They argue they might revisit them, they might need this data, someone
might take on a PhD student to trawl through it,
or you are in the movie business and your movie has premiered yet
there is a directors cut scheduled for next year...
OK, so you can add more Panasas. Cue salesman buying in a large bucket
of glee to rub his hands in.
I agree may argument holds less water with Lustre.