[Beowulf] Parallel File system as a storage Solution

[Mark Hahn]

> I'm one of the PVFS2 developers -- just warning you that I might be a
> bit biased :>

I wonder if you could offer some comments on how you see PVFS2 vs Lustre.

> Typical /home directory usage patterns are quite different from
> typical other file systems.  Users have lots of small files
> (.dotfiles, directories, mail (if it's in maildir or mh format),
> source code).  They run 'ls' (so end up calling stat(2) all the time.  

yes - we actually force users to organize their files such that 
they only get a fairly small /home quota (~200MB), but tons of 
other space (/work (which is shared cross-cluster), /scratch, 
node-local /tmp, etc).  it's also worth pointing out that even the 
underlying raid structure for these filesystems should really be 
different: /home has fairly small blocks, for instance, so a big-stripe
raid5 is really a bad bet.  but /scratch can be raid0, and /work something
like raid 50...

or that /home makes effective use of the pagecache, and /scratch is 
basically write-through.

> I think you're on the right track here:  have an NFS-hosted /home
> directory where users can compile their applications.  have a pvfs2
> /scratch file system where those applications can write their IO when
> they run.

hmm.  it would be nice if FS's like Lustre and PVFS could be parameterized to
provide adequate /home performance as well, since it's not nearly as easy
to scale NFS.

anyone using fscache to speed up NFS in an HPC environment (even just 
for homes)?

thanks, mark hahn.