I've got 8 linux boxes, what now

[Greg Lindahl]

On Fri, Dec 07, 2001 at 01:48:59PM -0800, Martin Siegert wrote:

> At that time I had to make a decision about loadbalancing and a batch
> queing system. I decided to have none of it.

First off, you proceed to discuss why you don't like batch queues, but
don't talk at all about LVS and other techniques for load balancing.
That just means the user types "ssh cluster.sfu.ca" and they end up
logging into the node with the lowest load. That's very easy to use.

As far as your criticisms of batch queues, you don't have to set them
up that way. You can set it up so that all jobs run immediately. That
provides a load balancing function, and a central way to figure out
your job status. It doesn't provide ideal use of resources in the face
of oversubscription, but it can't be gamed by the users.

Alternately, you can provide a couple of scripts that do nothing but
(1) start a command line on the node with the lowest load, and (2) run
ps on all the nodes and grep for that user's username. Same
difference. The nice thing about the batch queue is that it also copes
with an MPI cluster in addition to a big pile of interactive nodes. Or
if you used Condor as your batch queue, you could add some desktop
machines to the cluster, for additional oompf at night.

greg