[Beowulf] cluster building for teaching (on the cheap)

[Geoff Jacobs]

Nathan Moore wrote:
> I'm planning to put together a cluster at my institution to serve a
> computational physics class and also to use for parallel code
> development and small-scale research.  My local land-grant university
> has plenty of horsepower available, so my primary goal is to set up a
> modest cluster that will "feel" like a big beowulf, and will allow my
> students to learn the basics of parallel programming (with MPI - the
> more execution threads the better).  Additionally, the system needs to
> be cheap (less than $5k).
> 
> After about 4 hours of diffusing around the web it has become clear to
> me that most (all?) commercial solutions are too expensive and I should
> try to put something together myself.  In browsing NewEgg last night one
> interesting solution was to set up 2  dual processor, dual core machines
> (ie 2 motherboards, 4 AMD 1.8GHz Opteron 265's, 8 total execution
> cores).  Spec'ed out (1GB Ram per core, P-ATA hard drives), this looks
> like about $500 per execution core.
> 
> I've never done this before and I'd appreciate your collective input:
> 
> (1) Does Linux/MPICH/gcc/g95 work pretty well with dual core opteron
> processors?
Yes, fine. I'm presently trying to track a problem in gfortran in one
application, but it seems to be a vast improvement on g77. A commercial
Fortran compiler will not cost very much (especially with edu discount),
but is often very useful for supporting existing code which is
non-trivial to adapt/debug for a new compiler.

> (2) Am I better off buying 8 of the cheapest Dells I can find and
> networking those together?
With X2 Athlon64s and Prescott-core P4s, there is a good selection of
really inexpensive dual core CPUs which can be mated with inexpensive
commodity motherboards. I think building white-box nodes would be your
best bet.

If you don't need the density afforded by server motherboards, nor the
ability to support high-speed interconnects like Myri or IB, going with
budget motherboards is an attractive option. Choice is, of course,
subject to questions of reliability for any particular model of board.

ECC ram is probably a good idea, though, so don't skimp when buying DIMMs.

> (2.5) Do you pay a premium for a 1-u or 2-u enclosure?
I don't think it's worth it at this size, on this budget. You'll
probably find your rack at the local home improvement store.

> (3) In general (processor type, peripherals held constant), is it
> cheaper to buy 2x standard processor boxes, 1 dual processor box, or
> half of a dual processor, dual core box?
Costs don't scale the way you think :)

> Other thoughts are welcome.
Check out the Value Cluster, although it's a bit outdated.
http://clustermonkey.net/content/view/41/33/

> regards,
> 
> Nathan Moore

-- 
Geoffrey D. Jacobs

Go to the Chinese Restaurant,
Order the Special