[Beowulf] PVFS or NFS in a Beowulf cluster?

[Robert Latham]

On Tue, Jan 18, 2005 at 06:50:30PM -0500, Mark Hahn wrote:
> > I don't see a contradiction to use both: NFS for the home
> > directories (on some sort of master node with an attached hardware
> > RAID), PVFS2 for a shared scratch space (in case the applications
> > need a shared scratch space across the nodes). 
> 
> that's certainly attractive.  has anyone tried PVFS2 in a *parallel*
> cluster?

Disclosure: I'm one of the PVFS2 developers.  

We deliberately designed PVFS2 to perform very well as a fast shared
scratch space.  Let NFS do what it was designed to do -- serve home
directories.  

To answer your question, I regularly set up PVFS2 volumes on Argonne
clusters for benchmarking, testing, and experiments.  Ohio
Supercomputing Center has PVFS2 deployed on a pretty big cluster.
People are using PVFS2 and having good results.  If you try it out and
you find bugs or issues (and what software doesn't?), the mailing
lists have quite a few helpful people.  

If nodes in your cluster are acting as both IO nodes and compute
nodes, you will see a performance hit (I think it's small, but that's
somewhat objective).  We aren't shy about that fact

http://www.pvfs.org/pvfs2/pvfs2-faq.html#sec:howmany-servers

More PVFS2 information:
http://www.pvfs.org/pvfs2

==rob

-- 
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Labs, IL USA                B29D F333 664A 4280 315B