[Beowulf] Building new cluster - estimate

[Mark Hahn]

> considering of building/purchasing 40 node cluster. Before contacting
> vendors I would like to get some understanding how much would it cost. The

vendors have at least list prices available on their websites.

> 1. newest Intel quad-core CPUs (Opteron quad-core cpus are out of question
> due to ridiculous pricing), good balance of price/performance

at least for code which is relatively cache-friendly.

> 2. reasonably fast interconnect (IB SDR 10Gb/s would suffice our
> computational needs (running LAMMPs molecular dynamics and VASP DFT codes)
> 3. 48U rack (preferably with good thermal management)

"thermal management"?  servers need cold air in front and unobstructed
exhaust.  that means open or mesh front/back (and blanking panels).

> - 2x Intel Xeon E5420 Hapertown 2.5 GHz quad core CPU         :  2x$350=$700
> - Dual LGA 771 Intel 5400 Supermicro mb                                :
> $430

this is an eatx-format board with the vendor requiring >= 550W PSU, right?
you should definitely budget for the add-in IPMI card.

> - Kingston 8 Gb (4x2Gb) DDR2 FB-DIMM DDR2 667 memory    :  $360

wouldn't a 5100-based board allow you to avoid the premium of fbdimms?

> - WD Caviar 750 Gb SATA HD                                                :
> $110

I usually figure a node should have zero or as many disks as feasible.

> - 1U case including power supply, fans                                    :
> $150

I suspect this isn't a 550W EATX chassis...

> 48U rack cabinet, PDUs, power cables,                                  :
> $3000

seems like a lot - those must be pretty fancy controllable pdus.
for smallish clusters like this, I'd feel entirely comfortable with 
just IPMI, omitting controllable pdus.  (I would not omit ipmi in favor
of controllable pdus.)

> I would like to get some advice from experts from this list, is this pricing
> realistic? Are there any flaws with configuration?

the pricing is agressive; cost from a vendor like HP will be substantially
higher.

> In principle, we have some experience in building and managing clusters, but
> with 40 node systems it would make sense to get a good cluster integrator to
> do the job. Can people share their recent experiences and recommend reliable
> vendors to deal with?

HP is safe and solid and not cheap.  for a small cluster like this, 
I don't think vendor integration is terribly important.

regards, mark hahn.