[Beowulf] cluster building for teaching (on the cheap)

[Robert G. Brown]

On Sat, 26 Aug 2006, Nathan Moore wrote:

> I'm planning to put together a cluster at my institution to serve a 
> computational physics class and also to use for parallel code development and 
> small-scale research.  My local land-grant university has plenty of 
> horsepower available, so my primary goal is to set up a modest cluster that 
> will "feel" like a big beowulf, and will allow my students to learn the 
> basics of parallel programming (with MPI - the more execution threads the 
> better).  Additionally, the system needs to be cheap (less than $5k).
>
> After about 4 hours of diffusing around the web it has become clear to me 
> that most (all?) commercial solutions are too expensive and I should try to 
> put something together myself.  In browsing NewEgg last night one interesting 
> solution was to set up 2  dual processor, dual core machines (ie 2 
> motherboards, 4 AMD 1.8GHz Opteron 265's, 8 total execution cores).  Spec'ed 
> out (1GB Ram per core, P-ATA hard drives), this looks like about $500 per 
> execution core.
>
> I've never done this before and I'd appreciate your collective input:
>
> (1) Does Linux/MPICH/gcc/g95 work pretty well with dual core opteron 
> processors?

Yes.  However, YMMV depending on the application you try to run in
parallel and its communication/computation dependent scaling properties.
If the application is memory bound (does lots of streaming access of
memory) dual cores share a memory bus channel and they "can" compete for
bandwidth, reducing the potential parallel speedup.

For a bunch of CPU-intensive embarrassingly parallel applications, or
even for real parallel applications without too severe a streaming
memory requirement they should do fine.

A second place to think of bottlenecks is the network, see below.

> (2) Am I better off buying 8 of the cheapest Dells I can find and networking 
> those together?

For instructional purposes, this would be just fine, perhaps even ideal.
If you put them all on (say) a gigabit ethernet switch you could easily
demonstrate parallel speedup and Amdahl's Law type scaling to saturation
and eventual slowdown with toy code that lets you adjust the
compute/communicate ratio.

It also give every CPU its own memory bus, and its own network
channel(s), with no possibility of competition.

However, the opterons are likely to be faster in many ways, per core.
Even here YMMV and the best thing to do is to prototype a unit or two
and run your favored application(s) on it as benchmarks.

> (2.5) Do you pay a premium for a 1-u or 2-u enclosure?

Compared to OTS towers, yes.  It isn't that big anymore, though.  It
really depends on who you get them from and what you get.

> (3) In general (processor type, peripherals held constant), is it cheaper to 
> buy 2x standard processor boxes, 1 dual processor box, or half of a dual 
> processor, dual core box?

This is what google and pricewatch are for.  I haven't really priced out
dual core systems in comparison to dual processor single cores, but two
single processor systems are very likely the most expensive of these
three options per core.  The biggest difference being the replicated
case and peripherals.

     rgb

>
> Other thoughts are welcome.
>
> regards,
>
> Nathan Moore
>
>
> - - - - - - - - - - - - - - - - - - - - - - -
> Nathan Moore
> Physics
> Winona State University
> AIM:nmoorewsu
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu