newbie: 16-node 500Mbps design

[Laurent Itti]

Hi all -

I will soon start a computational neuroscience lab at USC, and would
like to install a small 16-node Beowulf system there.  Applications
will be either running multiple simulations of a same program with
different parameters (each run taking a few days), or processing video
sequences through the entire cluster, each node taking care of some
aspect of visual processing (e.g., edge detection, color processing).

For the latter application, high network throughput is needed, so I am
thinking of channel-bonding 5 cheap ethernet boards per node, all
connected through 5 switches. We have about $20k to spend on the baby,
so we target about $1k/node plus switches and misc.

For that price, we target 733MHz/256Mb/30Gb per node.

To be open to upgrades and recycling of nodes as desktops when faster
nodes are introduced, we take mobos with integrated video and sound.

I would greatly appreciate comments on the following:

- PIII/733 vs. Athlon/800 is a hard choice.  The i815 chipset for PIII
  seems to be supported under Linux, delivering full ATA/100 speed
  (www.gentus.com).  I have not found the equivalent for Athlon mobos.
  Is there any Athlon mobo with integrated video/sound?

- has anybody ever benchmarked ethernet boards under Linux? Why pay
  $50+ for a 3com board when a good old RTL8139 sells for $8?  Does
  anybody have an idea of possible differences in throughput or CPU
  use (using Linux drivers, not windoze or theoretical)?

- 16 singles vs. 8 dual-cpu nodes is also a hard call. With 16 singles
  we can get better distribution of data flow over the net. But I am
  open at this point to suggestions regarding good dual-CPU boards
  with full Linux support.

- I am unclear at this point as to whether the Abit SE6 mobo does have
  the integrated ethernet. Can't find those CNR riser boards for sale
  anywhere!

In addition to the 16 compute nodes, we will have a desktop (in
another room) to control the whole thing. The following is my draft
starting point to begin negotiating with local retailers. Any comment
also appreciated!

Thanks a lot!

  -- laurent itti


Seventeen (17) nodes consisting of:

Abit SE6 motherboard (i815E+ICH2 chipset)		$145	gogocity
CNR riser board						???
Intel 733EB FC-PGA CPU (retail box version?)		$240	egghead
Qty. 2, 128Mb PC133 168-pin SDRAM			$270	local
30Gb UATA/100 7200 RPM Hard drive			$170	egghead
Mitsumi 1.44 floppy drive				$10	local
Qty. 4, RTL8139 10/100 ethernet card			$32	local
Qty. 5, 3' enhanced CAT5 350MHz ethernet cable		$2.50   kristamicro
Qty. 1, Panasonic Panaflo fan, 500,000 hrs MTBF		$3	dealdeal
ATX desktop case					$45	???
Mandrake Linux 7.1 deluxe				$56	linuxmall
--------------------------------------------------------------------
					     Qty. 17 @  $973.50 each

[the 17th node, aka the "control node" (to be used as a regular
desktop PC) will actually have only one (on-board) ethernet, but two
longer ether cables; see below.  It will share keyboard/mouse/monitor
with another PC already in place, using a 2-port KVM switch. Five
switches will be used to connect the beowulf: 4x16-port and 1x24-port,
as the latter will also connect the beowulf to the outside].

[I like Mandrake 7.1 and will patch the kernel (if necessary) using
stuff from www.gentus.com for ATA/100 support. Intel has X servers].

17 nodes						$16,549.50
delete Qty. 4 RTL8139 and Qty. 5 ether cables	       -$34.50
7' ether cable, control node to wall			$2      kristamicro
25' ether cable, wall to beowulf			$5      kristamicro
Qty. 4, Linksys Etherfast-II 16-port 10/100 switch	$1,200	egghead
24-port Linksys Etherfast-II 24-port 10/100 switch	$427	egghead
Belkin Omnicube PS/2 2-port KVM switch			$90	egghead
Qty. 2, Belkin gold KVM cable kits			$50	local
Seagate Scorpion 4/8 internal IDE tape drive		$190	local
Qty 10, 4/8 tapes					$250	local
---------------------------------------------------------------------
							$18,729
			CA/LA County sales tax @ 8.25%	$1,545.14
							-------------
							$20,274.14

[almost all disk space is /tmp, so no need for big tape drive]