[Beowulf] Distributed Parallel Caching Filesystems

[hanzl at noel.feld.cvut.cz]

> 	I am looking around at high-performance i/o environments ...
> ... a clustered file system that can take span multiple I/O 
> servers with multiple network interfaces and cache to local systems 
> based on locality of data ...  Coda seems to have some of 
> this functionality, but I really don't want to make users have to 
> explicitly send and release data, and I do need standard posix 
> semantics. ...

I think NFS with fscache could provide much of Coda functionality in
very cluster-friendly manner. It is still in testing phase but I
consider it one of the 'strategically safest' options for these
reasons:

 - it could get to 2.6 mainline kernel

 - many non-cluster users will be also happy with it

There are (and was) many other projects going this way but quite often
there are reasons to be very skeptical about their long time viability
or even their actual existence (I mean existence of anything usable
behind the hype).

It is possible for a project to make headlines, be recommended over
and over again and yet have no single user who would attest that it
works for him.

It is possible to make quite interesting and working kernel
modifications going the way you described but it is much harder to
keep the thing alive when kernel changes.

So, to name something usable I trust ... PVFS comes to my mind, but it
might not have all the functionality you want (but it has very nice
and friendly team of developers) ... I do not recall much else, others
please help me...

Regards

Vaclav Hanzl

http://www.redhat.com/archives/linux-cachefs/2004-October/msg00027.html
- 2.6.9-rc4-mm1 patch that will enable NFS (even NFS4) to do
persistent file caching on the local harddisk

http://www.redhat.com/archives/linux-cachefs/2004-October/msg00004.html
- older message explaining what is going on

http://www.redhat.com/archives/linux-cachefs/2004-October/msg00019.html
- about ways to get this to the mainline kernel

http://www.redhat.com/mailman/listinfo/linux-cachefs
- list archives and subscription page

https://www.redhat.com/archives/linux-cachefs/2004-November/msg00005.html
- patch against vanilla 2.6.9 to try it out