[Beowulf] Distributed FS (Was: copying big files)

[Mark Hahn]

>> Extending your storage that way is just darn cheap.

in fact, we really have to stop thinking of storage as a significant
cost component to clusters.  if adding a terabyte disk to a node 
increases its cost by ~2%, then it's in the noise.

> Take a look at PVFS (www.pvfs.org).  Disclaimer: I work on PVFS, but
> seriously, PVFS has been used this way for a decade.

the premise of this approach is that whoever is using the node doesn't
mind the overhead of external accesses.  do you have a sense (or even
measurements) on how bad this loss is (cpu, cache, memory, interconnect
overheads)?  if you follow the reasoning that current machines are 
pretty 'fat' wrt IB bandwidth and cpu power, there's still a question
of who does the work of raid/fec - ideally, it would be on the client
side to minimize the imposed jitter.

thanks, mark hahn.