A beowulf for parallel instruction.

[Robert G. Brown]

On Wed, 1 Nov 2000, J, A. Llewellyn wrote:

> We are planning to provide a vehicle for instruction in
> parallel computing. Given this rather vague objective,
> what influence might this have on the selection of
> hardware and software? The selection of non-bleeding
> edge components is one obvious issue but there must be
> others.  Although this is slightly off-topic I'd
> greatly appreciate comments from those who are running
> machines that may cater to this area as part of their
> workloads.

One suggestion is to focus on collecting sets of building blocks.  To
illustrate the various tradeoffs, for example, it might be useful to
have 8-16 nodes that could be rewired and reconfigured rather freely
with and without heads and isolation into "Scyld beowulf", "NOW cluster"
(with heads on all nodes and real or simulated interactive loads),
"isolated cluster" (headless nodes but with full linux and interactive
capability) and so forth.  Assemble a nice little pile of NICs (perhaps
as many as 4/node in a couple or three flavors) so you can demo and
benchmark the effect of e.g. channel bonding, building hypercubes vs
other topologies.  If you have the money, get gigabit ethernet and/or
Myrinet for the nodes.  If you REALLY have the money, get a small set of
alpha nodes (perhaps 4-8), a small set of Intel nodes (4-8) and a small
set of Athlon nodes (4-8) so that you can REALLY investigate how these
various design decisions influence the efficiency and scalability and
cost/benefit of the final designs.  

For the most part I'd avoid rackmount designs simply because rearranging
and reconfiguring will be a bit easier if you have a small stack of
tower units on shelves open at the back.  You might even consider
putting all the units on tables with heads attached and with a whole
bunch of wires running along the back -- the same units can serve as
"desktops" during classes and "nodes" during the problems with a little
plugging and unplugging in the back.  Of course if you are rich enough,
you can have a dedicated rack AND a pile of desktop nodes and do both.

Get several different KINDS of switch for at least the 100BT stuff
(where it is relatively cheap).  Measure and illustrate the performance
differences associated with store-and-forward and cutthrough and with
stackable vs all-on-one-crossbar for various problem topologies and
communications patterns.  In addition to teaching about beowulfery, you
can do really valuable research with such a laboratory and your students
can publish the results and present them at linux and HPC conferences.

It would be such fun to have a laboratory like this available for
general use (over the network, by arrangement) within the beowulf
community.  Perhaps one way e.g. Scyld could think about making money is
to construct such a laboratory and sell benchmarking services and give
classes -- companies, at least, are often willing to pay thousands of
dollars to put their people through training courses and even
Universities would in many cases pony up lesser amounts.  The
comparative benchmarks and positive objective/quantitative answers such
a "public beowulf laboratory" could provide to various list questions
would also be invaluable.  Finally, it would be very nice to have a
place that offered up benchmarking services for free (to research
groups) or for a nomimal fee (to corps) -- oft-repeated advise on this
list is "running your application on the hardware you are thinking of
using is the only meaningful benchmark" but this is easier to tell some
one than arrange in reality.

We're close to having such a spanning set of Linux-based hardware at
Duke -- I have 4 XP1000's, 16 933 PIII's (both single and dual, with two
flavors of RAM), piles of single and dual PII's and PPros, a mixture of
fairly new Athlons in several clocks, single and dual Celerons, and a
few different network switches, and it is VERY instructive to design and
run various benchmarks and observe differences in performance.  We still
don't have gigabit networks (our problems don't really justify the cost)
but one day I hope to.  Also, we don't teach courses in beowulfery (we
are a physics department after all:-) although perhaps we should...;-)

Good luck, and have fun.  Let me know if/when you want more "advice" on
what to teach with whatever you build.  I have some pretty definite
ideas and even a few illustrative tools to contribute.

    rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu