[Beowulf] best archetecture / tradeoffs
Robert G. Brown
rgb at phy.duke.edu
Tue Aug 30 17:17:42 PDT 2005
David S. writes:
> A clever way I've seen of doing NFS-root uses AFS instead, with a
> replicated root volume mounted read-only on the clients. AFS' local
> caching makes it fairly efficient, and replicating the volume means
> the server is no longer a single point of failure. As long as one
> server hosting the volume is available, the clients will keep running.
> I've considered using AFS in place of NFS in clusters, but have been
> been discouraged by the diffculty of managing tokens and Kerberos
> tickets has discouraged. AFS-root is pretty slick, though.
> David S.
This is a joke reply, right:-)
Let's see. Yes, there are tokens, which often expire mid-computation.
There is kerberos, a lot of overhead indeed in a firewalled private
internal network. There is AFS's "I didn't really mean it" attitude
towards implementing e.g. fflush on mounted AFS volumes, where writeback
occurs only when a file is closed (making it impossible to use shared
files across nodes for a variety of purposes without adding all sorts of
file stats and overhead).
The one think I do like about AFS is its "real" ACLs. Unix's file perms
suck as a mechanism for enabling shared/group work, although they do
give sysadmins that warm fuzzy feeling of approximate control.
Seriously, we've kicked AFS around as a cluster FS before here, and I've
even used it in some computations I did a decade or so ago (where I
learned the hard way about the hahaha attitude towards implementing
fflush()) but it doesn't seem to be robust or at all easy to manage for
this sort of thing -- overkill in one place (security), underkill
someplace else (behaving like a rational/reliable filesystem).
Although RO AFS -- that one I haven't heard of, and obviously fflush
problems are irrelevant. So it might not be a joke after all:-) Don't
you need a pretty special kernel to make that work?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 189 bytes
Desc: not available
More information about the Beowulf