[Beowulf] diskless cluster nfs

[Tim Mattox]

Hello Josh,
We used to run our diskless clusters with hacked-together root
over NFS, but have switched to a more manageable system
based on ramdisks, called Warewulf.

Using the Warewulf cluster management tools, the contents of
the root filesystem on the nodes is easily maintained, updated,
changed, tweaked, customised, and whatever else you care to
do with them.  And in your particular case, there can be a
significant reduction in the NFS traffic on your cluster's network,
since all critical system files are loaded into RAM on the nodes
at boot time.

Since you are just getting started, it is probably worth your time
to check out at least one of these cluster management
tools for "diskless" clusters:
Warewulf  http://warewulf-cluster.org/
OneSIS  http://onesis.sourceforge.net/
They can save you a lot of headaches later.

Disclaimer: I liked Warewulf so much, I became one of it's
developers earlier this year.  I have never used OneSIS, but
it appears to be a viable approach if you really like root over
NFS.  In your case, I would suspect you would want to reduce
any extraneous NFS traffic as best you can.

Good luck.
-- Tim

P.S. - Take a good look at RGB's advice about the
scalability of your problem before you bang your head
against the "performance wall" for very long.

P.P.S - I sometimes think that NFS actually stands for
Not a File System... ;-)  It has no direct way to force a "sync".
I'd be very wary of any messaging scheme that used
NFS as the medium.

On Tue, 7 Dec 2004 17:23:40 -0500, Josh Kayse <josh.kayse at gmail.com> wrote:
> Ok, my first post, so please be gentle.
> 
> I've recently been tasked to build a diskless cluster for one of our
> engineers.  This was easy because we already had an image for the set
> of machines.  Once we started testing, the performance was very poor.
> Basic setup follows:
> 
> Master node: system drive is 1 36GB SCSI drive
>                      /home raid5 5x 36GB SCSI drives
> Master node exports /tftpboot/192.168.1.x for the nodes.
> 
> all of the nodes are diskless and get their system from the master
> node over gigabit ethernet.
>  All that worsk fine.
> 
> The engineers use files over nfs for message passing, and no, they
> will not change their code to mpi even though it would be an
> improvement in terms of manageability and probably performance.
> 
> Basically, my question is:  what are some ways of testing the
> performance of nfs ande then, how can I improve the performance?
> 
> Thanks for any help in advance.
> 
> PS: nfs mount options: async,rsize=8192,wsize=8192,hard
>        file sizes: approx 2MB
> --
> Joshua Kayse
> Computer Engineering

-- 
Tim Mattox - tmattox at gmail.com - http://homepage.mac.com/tmattox/