Need comments about cluster file systems

[Walter B. Ligon III]

--------

> > > (For sure others will point you to PVFS, which IMHO makes sense only
> > > if network card is quicker than local disk.)
> > 
> > It's frustrating to hear people talk about how wonderful InterMezzo and
> > Lustre _are_, and dismiss PVFS and GFS.  Software that is not quite
> > finished is always better and faster than software that already
> > exists.  It only loses speed and features when reality looms.
> > 
> > Talk about vaporware and deployed systems in different categories unless
> > you clearly use the future tense.
> 
> I quite like PVFS, but I think it does not solve the problem. AFAIK it
> can get speed of NIC. I want the speed of local harddisk, which is
> much bigger with my hardware. But again, I am ready for any
> enlightenment.

Well, the point is there isn't one kind of file system that serves everyone's
needs.  If your application is such that you know where you data needs to be before hand and you can run your computations on the same node, then you don't need anything more than the local file system.  If you need to be able to access any data from any node with a parallel application, that is what PVFS was designed for.  If the data you need is on the local disk, PVFS gives you local disk speeds.  If its not, you are limited by the network speed.  There
is no way around that.

The original poster who started this didn't specify what kind of applications
he was considering, thus it seems rather impolite to respond assuming that
HIS needs are the same as YOUR needs - especially when that involves trashing
someone else's work.

The original poster said something about his experience in the world of
supercomputers and MPPs there were parallel file systems, so one MIGHT think
that in fact he DOES have applications relavant to PVFS - but who knows, maybe
he'll tell us.

In the mean time, since there aren't any good open source solutions to the cached file system problem, why don't you work on it yourself.  Our newest version of PVFS is designed so that things like that can be added as modules.
We don't have the manpower to solve ALL of the problems, so we are trying to
at least create a framework for collaboration.

Walt

-- 
Dr. Walter B. Ligon III
Associate Professor
ECE Department
Clemson University