[Beowulf] Diskless booting - NIC BIOS

[Robert G. Brown]

On Mon, 18 Jun 2007, Ellis Wilson wrote:

> Thanks Brian, Matt, and rgb,
>
> The floppy idea is great (I think I remember now reading about it in
> rgb's book, but had forgotten), and I certainly will look into that.
> The motherboards are in some cases years and years old; one computer I'm
> deciding whether I'll use or not does have a 400mhz processor in it, so
> their age is sufficient to make me worry.

Two points.  One is that these days if your system has a BIOS that can
manage booting from CD, I'd advise booting from CD instead of floppy.
There are a variety of reasons for this -- CD's are cheap, you can put a
large kernel on it, you can actually put a whole linux image on it and
avoid having to "boot diskless" over the network, although of course you
can still do that as well.  Floppies are pretty much obsolete at this
point and it isn't easy to get a properly bootable image of a modern
kernel to live on one -- I think you'll find building tight kernels that
will fit moderately frustrating.

Second, remember that one dual dual core 64-bit opteron processor system
-- currently available for maybe $1600 if you shop hard -- is going to
be faster than a 32 node 400 MHz P6 cluster, and the latter will cost
around $3000/year to leave powered on 24x7 (estimate $1/watt/year, even
if you're not paying for it...:-). So you're building your cluster to
learn and have fun, not for speed or to save money.  If you have real
work to do and want to do it as cheaply as possible, it would be wiser
to go with a very small cluster of dual core 64 bit modern CPUs.

> In response to how I'm handling "DHCP/TFTP management", I am
> ridiculously interested in knowing how everything works, so I install
> each computer using a method that follows the guidelines similar to
> Linux From Scratch.  I'll likely be using dhcpcd for DHCP and I'm
> currently looking into TFTP options.  If there is a suggested program of
> choice among you all, please feel free to let me know.  I opt for the
> Linux from Scratch route, one, because I hate going to class and thereby
> have plenty of time on my hands (hah), and two because the majority of
> computers I deal with are old and need all the free resources they can
> get.

I personally would strongly advise a newbie learning about clusters to
go on one of three routes these days.  Route one would be warewulf --
this project comes with diskless booting more or less directly supported
(in a sense, the core of the project IS a diskless distribution system
for node images) and is distribution neutral, within reason.  There are
Real Humans using it and mailing lists and so on to support it, both
important.  Routes two and three would be to go either Debian or Fedora
Core -- both have advantages (and no, I'm not getting myself trapped in
a religious debate over which one is is good and which one is evil -- I
actually think both are pretty good) and disadvantages.

Setting up for diskless booting is actually pretty easy with the stock
tftpd and dhcpd from FC (the distro I generally use these days).  There
is plenty to learn using one of the high end distros without quite
having to build everything yourself, which can be frustrating as much as
illuminating, and there are some WONDERFUL tools in the newer bleeding
edges of these distros.

One other thing to play with that can suck you right in but that should
prove to be very rewarding in the future is virtualization -- look over
vmware-player and the library of VM appliances, including prebuilt
ready-to-play cluster nodes.  Xen promises to be similarly useful
although so far it appears to me to be more cumbersome and less stable
when supporting a workstation as opposed to a stripped down server
image.  Might be very good for stripped down node images, though.
Virtualization lets you REALLY play with your system(s).  You can
actually boot (say) FC and run (say) Debian and even (yuk) Windows in
two VMs and switch freely between all three images in multiple
workspaces.  Tres cool.

    rgb

> Again, much thanks to those who helped.
>
> Ellis
>
> Brian Dobbins <brian.dobbins at yale.edu> wrote:
> Hi Ellis,
>
>  I wasn't sure from your post whether you meant the nodes had /no/
> network whatsoever, or simply no capabilities for network booting from
> the NICs in the system.  If it's the latter, and assuming these systems
> have a floppy drive, I'd suggest looking into using the Etherboot
> software to handle network booting.  No need to spend extra money.  :)
>
>  The webpages will explain more, but essentially (from memory - it's
> been a while!), if you set up a DHCP / TFTP server for the images
> somewhere on the network, just create a boot floppy with the correct
> network drivers for the node, stick it in, power on, and provided the
> DHCP/TFTP servers are correctly configured, the node should boot up,
> initialize the network, send out a request to the DHCP server, and then
> (from the information handed back), request a boot image from the TFTP
> server.  To create this boot floppy, you can probably just visit the
> Rom-O-Matic page ( http://rom-o-matic.net/ ) and select the type of card
> you have, but definitely read over the Etherboot documentation, too (
> http://www.etherboot.org/ ).  If you're not certain what type of card is
> in the nodes, I'd suggest putting a Knoppix CD in, booting up, starting
> the network,  and then listing the modules that are loaded - the network
> drivers should be in that list.
>
>  If you get stuck, drop me a note and I'll be glad to try to walk you
> through it - I'm a pack rat, and probably still have all the old
> configuration files from when I last did this, too.  Finally, in terms
> of the DHCP/TFTP management, are you handling that by yourself, or using
> some already-written package?  The initial cluster that I used Etherboot
> on used the Warewulf package - I'd recommend you take a look at it,
> too.  The webpage is ( http://www.warewulf-cluster.org/ ).  The guy
> developing it, Greg Kurtzer, is really helpful, too, so if you get stuck
> in that stage of things, you won't pull out all your hair in frustration.
>
>  Good luck!
>  - Brian
>
> (Naturally, RGB also helped me out in the past -- anyone know if there
> is some analogue in the Beowulf realm to the Erdos number for RGB?  I
> can't imagine there's anyone he /hasn't/ helped!)
>
> Brian Dobbins
> Yale Engineering HPC
>
>
>
> ---------------------------------
> Fussy? Opinionated? Impossible to please? Perfect.  Join Yahoo!'s user panel and lay it on us.

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu