[Beowulf] motherboards for diskless nodes

[Robert G. Brown]

On Fri, 25 Feb 2005, Greg Lindahl wrote:

> On Fri, Feb 25, 2005 at 06:52:59PM +0100, Ryan Sweet wrote:
> 
> > While I can understand debating over the merits of nfsroot vs RAM-disk 
> > root, I fail to see many useful arguments for maintaining a local OS 
> > install.
> 
> An example of something that goes very wrong with NFS is upgrading a
> file to a new file with the same name. If that file is a binary or
> library that's in use anywhere in the cluster, you are likely to have
> a problem. Local disks and Scyld, on the other hand, do the right
> thing: existing processes using the binary or library continue to use
> the old version, while new ones use the new version.
> 
> This disagreement is as old as the hills, by the way: in the good old
> days, when Sun was young, lots of people ran their pizza-box
> workstations diskless, but that went out of style when Ethernet's
> performance was stuck in place for a bunch of years.
> 
> It's important to understand arguments you disagree with; your
> dismissal is not a good sign.

To add to Greg's remarks, yes diskless is a perfectly valid way to
structure a LAN (and I'm, alas, as old as the hills and actually ran
whole networks with sparc 1's, SLC's and ELC's diskless.  They typically
only had 4 MB or so of RAM (in later days as much as 16 or 32) and
actually did remote swap as well as NFS home directories and binaries
and so forth.  They worked amazingly well given the times.

The notion of "thin clients" goes back even before Sparcs -- I once ran a
wierd IBM PC clone in the mid-80s that had a proprietary "network"
interface, a hook into the bios of a host PC, and ran "diskless" by
leeching on the floppies and hard drive or the host.  It was marginally
cheaper than getting a regular PC with its own hard drive because back
then disk cost a mint -- all peripherals cost a mint.  PC's cost
thousands of 1980's dollars.  

I've run diskless linux clusters off and on as well, mostly from
necessity.  If you have enough memory it's good.

Still, I actually think that there are excellent reasons to consider and
perform installs to local disk.  Robustness, speed, ease of installation
and maintenance with tools like PXE, kickstart and yum have taken just
about all the sting out of it.  Having local swap is good.  Having local
scratch is good.  Decreasing memory occupancy may be good -- having
local disk means local paging is possible with a small performance edge
(depending on your network and so forth).  Thin clients have been
proposed periodically over the years, but they never quite take off --
it is just too damn convenient to have some measure of local robustness,
and hard disks are cheap.

> 
> > It can be _very_ useful in a situation where you have to support multiple 
> > user communities with wierd apps or strange requirements.
> 
> Yep. But your conclusion:
> 
> > That sort of config requires 
> > a lot of glue, but it would be way too much headache to even attempt 
> > without "diskless" systems.
> 
> Doesn't make any sense; I have seen people describe such systems where
> they download a disk image when a batch job wants a different software
> load. It's certainly doable that way: it does have different tradeoffs
> from the diskless case, but if it gives you a headache, it's probably
> because you don't like it, not because it's hard to do.

Ya.  Right now I think it is kinda cool just how MUCH one can do, all of
it pretty easy.  In the old days it was God's Own PITA to set up
diskless anything -- I wrote all kinds of stuff myself to get systems to
boot diskless and at least I COULD do it because I'd run Suns from the
old days, and then there was management of "packages" (binary
architectures and shared this and that) with no particular organization
on top of that. You young'uns have it all easy -- several different
toolsets to choose from to set up diskless operation, several different
toolsets to choose from to set up disk and manage packages, and far more
homogeneous hardware.

I certainly don't think that diskless is a knee-jerk obvious choice for
either LANs or clusters, although sure, there are some advantages to it
(just as there are some advantages to having local disks).


   rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu