[Beowulf] recommendations for cluster upgrades

[Tiago Marques]

Hi,

I'm part of a - still small - computational group at Aveiro, Portugal, and
we only have 8 dual quad-core 2.66GHz Xeons.

I've benched a Core i7 with some common computation and it's incredibly fast
compared to Xeons 5400.

One of the codes, VASP, is very bandwidth limited and loves to run in a
number of cores multiple of 3. The 5400s are also very bandwith - memory and
FSB - limited which causes that they sometimes don't scale well above 6
cores. They are very fast per core, as someone mentioned, when compared to
AMD cores.

These are the times I get from a benchmark I usually run in VASP:

VASP on Core i7:
        - 1 core = 162.453s, 162.778s (no HT)
        - 2 cores = 100s,102s (no HT)
        - 3 cores = 77.835s, 78.195s (no HT)
        - 4 cores = 87.63s, 87.322s (no HT)
        - 6 cores = *76.56s, 76.4s*
        - 6 cores DDR3-1600 CAS9 - 69.654s, 68.816s, 67.7s

HT doesn't add much but DDR3-1600 does. Still, ~78s is very fast with a
quad-core because our dual 5400s can only do *91s* at best, even using
tweaks like CPU affinity, which brings it down from 95s, by distributing
only 3 threads per socket and not 4/2 or having 4 of them constantly jumping
from socket to socket.

Also, the Core i7 run GotoBLAS prepared for Penryn CPUs and IFort compiled
VASP with all the optimizations for the Penryn Xeons. Compiling both for
Nehalem would probably yield 10-15% more performance.

A desktop quad-core trouncing a one year old dual Xeon QC left me thoroughly
impressed. It will also scale better if you can use the extra cores
available, while this has rarely been the case with Penryn based Xeons - at
most it gave us 5% improvement of 6 cores.

Unfortunately I didn't have time to run anything more and VASP is our main
"customer".

Still, if you can upgrade to Shangai cpus, it may be a better deal than
switching all to Nehalem.

Best regards,

Tiago Marques



On Tue, May 12, 2009 at 8:19 PM, Rahul Nabar <rpnabar at gmail.com> wrote:

> I'm currently shopping around for a cluster-expansion and was shopping
> for options. Anybody out here who's bought new hardware in the  recent
> past? Any suggestions? Any horror stories?
>
> We've been using Dell SC1435's with Quad-Core AMD 2354 Opterons @
> 2.2GHz. 16 Gig RAM.
>
> Any new-cutting edge stuff I ought to be asking my vendors to put into
> the quotes? I already have gigabit bonded backbones and don't think we
> have the financial muscle to upgrade to Myrinet or Infiniband yet. In
> the interest of homogeniety and not wanting to have dual trees of
> executibles I might be tempted to stick with AMDs unless there is
> compelling temptation otherwise.
>
> I am already looking at the CPU benchmarks on the Intel/AMD websites
> but they can sometimes be misleading / misrepresenting other than the
> obvious glaring conflict of interest. I rather trust first-hand
> anecdotal evidence from you guys actually administering them for
> scientific applications. Unless there is a good third party, relevant
> database?
>
> For some reason the top500 sublists seem skewed to prefer the Intel
> Xeons. Why so few Opterons or any other AMD hardware? Just curious if
> this is driven by technological inferiority of only a marketing
> effect. My vendor seems to be trying to steer me towards an Intel
> Nehalem or Clovertown for whatever reasons good or bad.
>
> Ultimately of course, it might be best if I just got to benchmark my
> very own application on these CPUs before I bought them. But that's
> just wishing I guess!
>
> --
> Rahul
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20090516/0e9559da/attachment.html>