[Beowulf] Which is better: 32x2 or 64x1

[Andrew Shewmaker]

On 4/26/06, hpc at gurban.org <hpc at gurban.org> wrote:
> Hi,
>
> I'm going to build a beowulf linux cluster for my
> coledge. Thay want 64 core cluster with Opteron
> processors.
> Now I want to know which is better (performance & price)?
>   * 64 x Opteron Single Core
>   * 32 x Opteron Dual Core

The memory performance of Opteron Dual Cores is less than Single cores
in a cluster because there are fewer memory controllers and they will
experience more contention.  Dual Cores are also generally slower, so
they have a lower theoretical performance and their integrated memory
controller is slower.

Opteron Dual Cores have a higher memory latency than single cores (5-6
ns for single and dual socket systems) .  That is much less than the
increase in latency seen by accessing memory over Hypertransport
(roughly 30 ns).  Note that Hypertransport is still very good.

http://www.amd.com/us-en/Processors/ProductInformation/0,,30_118_8796_8800~97042,00.html

The fourth page of the following document has a nice table that shows
what kind of Stream performance you can expect with different numbers
of threads using dual and single core Opterons.  Each thread gets
fewer MB/s, but Dual Cores with more threads can achieve a greater
aggregate rate.

http://www.amd.com/us-en/assets/content_type/DownloadableAssets/AMD_Opteron_Streams_041405_LA.pdf

Another thing to consider is the quantity of memory you are able to
purchase and fit in your cluster.  Because you will have half the
number of memory controllers, you can have half the amount of memory. 
With Single Cores you could also buy less dense, cheaper memory while
still getting the same total amount.

If your code is CPU or cache bound, or if this is just a cluster to
learn on, then I would go with Dual Cores because I would expect them
to be cheaper.  If your code is memory or network bound, or if your
code is limited in how far it can scale across processors; then I
would go with the faster Single Cores that will also have greater
memory and network resources per core.

--
Andrew Shewmaker