[Beowulf] diskless booting over jumbo frames

[Bogdan Costescu]

On Tue, 24 Apr 2007, Mark Hahn wrote:

> so the main question is whether jumbo is worth the effort.

I would rephrase it to say: whether jumbo is worth the effort for the 
root FS. When I used NFSroot, most of the traffic was queries of 
file/dir existence and file/dir attributes, which are small, so a 
large maximum packet size would not help. Furthermore, most of the 
files accessed were small which means that the client could be quite 
successful in caching them for long times and the actual transfer (if 
the cache is emptied) would not take too long.

I think that it is more important to think thoroughly the placement 
and exporting of the files on the NFS server. If you can manage to 
export a single directory which is mounted as-is by the clients and 
have the few client-specific files either mounted one by one or 
copied/generated on the fly and placed on a tmpfs (and linked from 
there), you can speed up the serving of the files, as the most 
accessed files will stay in the cache of the server. The Stateless 
Linux project from Red Hat/Fedora used such a system (single root FS 
then client-specific files mounted one by one) last time I looked at 
it.

> here's a cute hack: if you're using pxelinux, it has an "ipappend" feature,
> ...
> I haven't had time to try this...

It works as you described it.

But even the first idea that you mentioned, using dhclient to get an 
IP would work just as fine if the number of nodes is not too big - I 
have 100+ nodes configured that way, with 2 DHCP requests per boot of 
node (the PXE one and the dhclient one) as I was just too lazy to try 
to eliminate the second DHCP request by re-using the info from PXE - 
and the master node doesn't feel the load at all, although it is 
hardware-wise the poorest machine from the cluster (as opposed to most 
other clusters that I know of ;-)).

-- 
Bogdan Costescu

IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen
Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY
Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868
E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De