[Beowulf] Building new cluster - estimate

[Ivan Oleynik]

I am in process of upgrading computational facilities of my lab and
considering of building/purchasing 40 node cluster. Before contacting
vendors I would like to get some understanding how much would it cost. The
major considerations/requirements:

1. newest Intel quad-core CPUs (Opteron quad-core cpus are out of question
due to ridiculous pricing), good balance of price/performance
2. reasonably fast interconnect (IB SDR 10Gb/s would suffice our
computational needs (running LAMMPs molecular dynamics and VASP DFT codes)
3. 48U rack (preferably with good thermal management)

I used newegg plus some extra info to get pricing for IB. This is what I
got:

Single node configuration
------------------------------------------------------------------------------------------------------------------
- 2x Intel Xeon E5420 Hapertown 2.5 GHz quad core CPU         :  2x$350=$700
- Dual LGA 771 Intel 5400 Supermicro mb                                :
$430
- Kingston 8 Gb (4x2Gb) DDR2 FB-DIMM DDR2 667 memory    :  $360
- WD Caviar 750 Gb SATA HD                                                :
$110
-Melanox InfiniHost Single port 4x IB SDR 10Gb/s PCI-e card    :  $125   (
http://www.colfaxdirect.com/store/pc/viewPrd.asp?idproduct=12)
- 1U case including power supply, fans                                    :
$150
------------------------------------------------------------------------------------------------------

          $1,875/node

Cluster estimate:
------------------------
40 nodes:

: 40x$187=$75,000
2x24-port 4X (10Gb/s) 1U SDR IB Flextronics Switches            :
2x$2400=$4,800
(http://www.colfaxdirect.com/store/pc/viewPrd.asp?idcategory=7&idproduct=13)<http://www.colfaxdirect.com/store/pc/viewPrd.asp?idcategory=7&idproduct=13>
IB
cables
: 40$65=$,2600
48U rack cabinet, PDUs, power cables,                                  :
$3000
-----------------------------------------------------------------------------------------------------------
TOTAL: $85,400

I would like to get some advice from experts from this list, is this pricing
realistic? Are there any flaws with configuration?

In principle, we have some experience in building and managing clusters, but
with 40 node systems it would make sense to get a good cluster integrator to
do the job. Can people share their recent experiences and recommend reliable
vendors to deal with?

Many thanks,

Ivan Oleynik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20080723/08e8c4d7/attachment.html>