[Beowulf] Woodcrest Memory bandwidth

[Vincent Diepeveen]

Hi,

Unlikely that a single core is the problem.

Woodcrest 3Ghz we know little technical details though.

A few scenario's for memory reads to L1
as i didn't see public technical data from intel yet.

Most likely it has 1 port @ 2 cycles,
as its 'father' the Banias (hmm bit bitter that near this area named after 
Syrian part of Golan Height
such a 34 day war took place) can do the same.

If so in that case:

3Ghz * 8 bytes / 2 cycles = 12GB/s

This assumes of course that all other instructions don't stop its memory
bandwidth. As we know from article posted online by Johan de Gelas,
Woodcrest most likely prefetches memory operands already in the 64 bytes of
instruction cache that gets executed before it gets in execution unit.

So it should with even current codes have no problems obtaining that 12GB/s 
a core.

4 cores on paper could be 48GB therefore.

If we see numbers now of around 4 - 6.6 GB/s at 4 cores, that means that all 
4 cores are
near idle for sure and working at not even close to 10% of their maximum 
bandwidth.

So somewhere from the memory chipset to just before the sockets there is 
some bandwidth
problem.

Chipset of DDR2 ram. I'm not sure whether this is 128 bytes a clock. 
Additional i'm not so sure
how many clocks of the memory chipset is needed to get 128 bytes from the 
memory controller.

Others know way more here.

Some layman math:

Ok let's assume a cycle or 12 for ECC registered ram, at 667Mhz.

That's 0.667Ghz * 128 bytes / 12 = 7.1 GB/s bandwidth that the chipset can 
deliver.

The question is of course whether when it would be able to deliver more, 
whether intel lacks
hypertransport on its mainboards to get it to the processors.

Yet this all seems a simple memory chipset bandwidth limit to me.

For sure not a processor bandwidth limit.
That woodcrest processor is superior to any other processor on the planet.

Vincent

----- Original Message ----- 
From: "Joe Landman" <landman at scalableinformatics.com>
To: "Stu Midgley" <sdm900 at gmail.com>
Cc: <beowulf at beowulf.org>
Sent: Tuesday, August 15, 2006 1:10 AM
Subject: Re: [Beowulf] Woodcrest Memory bandwidth


> Hi Stu:
>
> Stu Midgley wrote:
>> sorry, forgot to reply all... don't you hate gmail's interface sometimes?
>>
>>
>> What is the memory latency of the woodcrest machines?  Since memory
>> latency really determines your memory bandwidth.
>
> Hmmm...  not for large block sequential accesses.  You can prefetch
> these assuming enough intelligence in the code generator (heh), or the
> hardware if the memory access pattern is fairly consistent.
>
> Latency really defines the random access local node GUPS, well, its
> really more complex than that, but roughly that.
>
> That said, I would like to measure this.  I have an old code which does
> this, any pointers on code other people would like me to run?  If its
> not too hard (e.g. less than 15 minutes) I might do a few.
>
>> If Intel hasn't made any improvements in latency then the limited
>> number of out-standing loads in the x86-64 architecture will limit the
>> bandwidth regarless of the MB/s you throw at it.
>
> Hmmm... Ok, you are implying that if your processor can consume the
> load/store slots faster than it can launch them, and there are a limited
> number of memory operations in flight (2? as I remember, not looking at
> my notes now), it is going to be load-store pipeline limited, not
> necessarily "bandwidth".  That is, the memory system would be
> significantly faster than the CPU can consume.
>
> I haven't looked closely at the Woodcrest arch yet.  Don't know
> precisely what they are doing here and how it differs from AMD.  Would
> be interesting.  So far I haven't been impressed with code that I
> thought I should be really impressed with on this machine.  Oddly the
> performance was about what we got out of the Core Duo on this platform.
>
>
>
> -- 
> Joseph Landman, Ph.D
> Founder and CEO
> Scalable Informatics LLC,
> email: landman at scalableinformatics.com
> web  : http://www.scalableinformatics.com
> phone: +1 734 786 8423
> fax  : +1 734 786 8452 or +1 866 888 3112
> cell : +1 734 612 4615
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf
>