[Beowulf] network filesystem

[Stephen Hocking]

>
> > Our developers  had  that issue  of inconsistent  file system  view in RHEL
> > based systems, some of it is solved by disabling dir list caching, another
> > by using noac,
>
> well, developers should be smart enough to know what FS they're using,
> and how it's intended to behave.  turning off AC is a nice option,
> but smarter is to leave it on and not try to cause race conditions.
> (I expect that such race-friendly behavior will fail on some other
> non-NFS filesystems, though probably harder to trigger.)

I recall doing a port of a former employer's seismic processing code
to Linux, which was used to having GPFS or PIOFS around. The only
distributed(!) filesystem of any sort that I could afford was NFS,
which wasn't too bad, except that various programs insisted on having
multiple nodes (say about 200+) appending to the same file
simultaneously. After much trial & error, I discovered noac and also
that opening the file on the clients with O_SYNC would send each write
off to the NFS server immediately. Not an elegant solution. All this
was happening over 100base-T, and the NFS server, if we were lucky,
had a GB connection to the switch. We discovered an interesting race
condition in one of the ethernet drivers along the way. They tell me
that they're using GPFS under Linux now.


    Stephen