[Beowulf] Storage and cachefs on nodes?

[hanzl at noel.feld.cvut.cz]

>> Anybody using cachefs(-alike) and local disks on nodes for
>> reboot-persistent cache of huge central storage?
>
> I can guarantee you are not the only one interested in this...
> ...
> ... Europen Bioinformatics Institute

Great, thanks. I always believed that this data access pattern must
appear in bioinformatics.

> ... we've been even semi-seriously thinking of implementing this
> ourself, but there is never enough time as usual.

People start to implement this again and again but none of the small
nice projects seems to survive in long term.

> We are looking for distributed filesystems (Lustre, Terragrid, ...)

Problem with most huge projects going this way is that they involve
special server while many users could be quite happy with just a
special client (NFS client with local filesystem cache and certain
degree of filesystem semantics screwup).


Most discussions on this topic end by "It can be done, if you need it,
just implement it". But the real question is how to implement it and
let it survive in long term - across changing kernel versions etc.

I think persistent file caching should be as independent as it can
get, using standard commodity server and being careful to minimize
dependencies in client. Solaris cachefs looked good from this point. I
am not sure how much can I expect from linux cachefs as seen in
e.g. 2.6.9-rc3-mm3 - if I got it right, it is a kernel subsystem with
intra-kernel API, being now tested with AFS and intended as usable for
NFS. It is however "low" on NFS team priority list. So linux cachefs
might provide cleaner solutions than Solaris cachefs - if it ever
provides them.

Regards

Vaclav Hanzl