[Beowulf] some tips on new cluster hardware

[Mark Hahn]

> 1) Suggestions on MoBo's that support a 1333MHz FSB to plug Intel's
> Q9450, with video and jumbo-frames capable Gigabit on-board;
> 2) NIC PCI Gigabit card, jumbo-frames capable (because I might be
> asking too much on item 1);
> 3) 16-port Gigabit Switch, jumbo-frames capable, with Flow Control.

it sounds to me as if you've done some research, and just want help shopping.
how thorough was your research?  often people start with preconceptions
that limit the range of solutions they look at, unnecessarily.  for instance,
you have clearly decided you want Gb+jumbo - how did you decide that?
jumbo is mainly about reducing the cpu overhead of high-bandwidth apps,
which does not necessarily correspond with typical cluster workloads.
if you're concerned about bandwidth, might you be better off with some 
other interconnect?  IB has gotten a lot cheaper recently.

to me, cluster hardware should really take one of two approaches:
high road or low.  the high road includes IB or 10Ge, and tends to 
also include rackmount and dual or quad socket.  this is approach 
provides an excellent platform for demanding (high-bandwidth and/or
tight-coupled) workloads.

the low road is more "beowulfy" - to leverage commodity parts.
that means gigabit, and excludes IB.  but it also means PC-style cases,
probably single-socket mATX boards with integrated video and gbit.
these parts are a LOT cheaper than high-road "server" boards.

main downsides of the low road:
 	- no real managability.  cheap boards don't have IPMI, though
 	nowadays they will at least PXE boot.
 	- at most 4cores and one memory controller per node.
 	- gigabit isn't exactly low-latency or high-bandwidth, so this
 	kind of cluster is mostly appropriate for serial or small/loose
 	parallel jobs.

> The most likely brands we can find here in Brazil are Intel, 3COM,
> D-Link, BCM and TrendWare.

I think you should look closely at using motherboard-integrated gigabit.
besides saving cost, it reduces the complexity of the system, and if you're 
feeling DIYish, you can mount boards caseless for extra savings (and often
better control of airflow!)

I also think you should avoid falling in love with your ethernet switch ;)
I just checked the price on the DGS-1216t, which appears to be what you're
looking for, and it's $Cdn 270 or so (cheaper than one cpu).  you should 
probably consider getting a modestly larger switch - 24 is still commodity,
and would let you expand or add a trunked uplink to fileserver(s).