[Beowulf] Woodcrest Memory bandwidth

[Steffen Persvold]

Not if you're using a "non-polluting" store (like movntps), but I don't know
if compilers are smart enough (they should be?).
 
Cheers,

--Steffen

> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org]
> On Behalf Of Keith D. Underwood
> Sent: Monday, August 14, 2006 10:48 PM
> To: Stuart Midgley
> Cc: Beowulf List
> Subject: Re: [Beowulf] Woodcrest Memory bandwidth
> 
> Um, I haven't looked closely at Woodcrest lately, but everyone does
> remember that on a write, you have to fetch the cache line that you are
> writing to, right?  So, if you have a 10 GB/s memory system, the most a
> copy should be able to do is:
> 
> read the source at 3.3 GB/s
> read the destination at 3.3 GB/s
> write the destination at 3.3 GB/s
> 
> What streams will report is 6.6 GB/s, which, um, matches the results
> earlier in this thread.
> 
> 					Keith
> 
> On Mon, 2006-08-14 at 19:00 -0600, Stuart Midgley wrote:
> > actually, latency determines bandwidth more than memory speed...  see
> > my rant(?) from a little over a year ago
> >
> > http://www.beowulf.org/archive/2005-July/013294.html
> >
> > prefetch etc. help, but the limiting factor is still latency.  Hence
> > the opterons have significantly higher real memory bandwidth (their
> > latency is about 1/3 that of xeons/p4's etc).  If you look at the
> > ia64's then they have even high latency again, but they can have a
> > huge number of outstanding loads (from memory its >80), so their
> > effective bandwidth is high.
> >
> > Stu.
> >
> >
> > On 15/08/2006, at 8:10, Joe Landman wrote:
> >
> > > Hi Stu:
> > >
> > > Stu Midgley wrote:
> > >> sorry, forgot to reply all... don't you hate gmail's interface
> > >> sometimes?
> > >>
> > >>
> > >> What is the memory latency of the woodcrest machines?  Since
> memory
> > >> latency really determines your memory bandwidth.
> > >
> > > Hmmm...  not for large block sequential accesses.  You can prefetch
> > > these assuming enough intelligence in the code generator (heh), or
> > the
> > > hardware if the memory access pattern is fairly consistent.
> > >
> > > Latency really defines the random access local node GUPS, well, its
> > > really more complex than that, but roughly that.
> > >
> > > That said, I would like to measure this.  I have an old code which
> > > does this, any pointers on code other people would like me to run?
> > > If
> > its
> > > not too hard (e.g. less than 15 minutes) I might do a few.
> > >
> > >> If Intel hasn't made any improvements in latency then the limited
> > >> number of out-standing loads in the x86-64 architecture will limit
> > >> the bandwidth regarless of the MB/s you throw at it.
> > >
> > > Hmmm... Ok, you are implying that if your processor can consume the
> > > load/store slots faster than it can launch them, and there are a
> > > limited number of memory operations in flight (2? as I remember,
> not
> > > looking at my notes now), it is going to be load-store pipeline
> > > limited, not necessarily "bandwidth".  That is, the memory system
> > > would be significantly faster than the CPU can consume.
> > >
> > > I haven't looked closely at the Woodcrest arch yet.  Don't know
> > > precisely what they are doing here and how it differs from AMD.
> > Would
> > > be interesting.  So far I haven't been impressed with code that I
> > > thought I should be really impressed with on this machine.  Oddly
> > the
> > > performance was about what we got out of the Core Duo on this
> > > platform.
> > >
> >
> >
> > --
> > Dr Stuart Midgley
> > Industry Uptake Program Leader
> > iVEC, 'The hub of advanced computing in Western Australia'
> > 26 Dick Perry Avenue, Technology Park
> > Kensington WA 6151
> > Australia
> >
> > Phone: +61 8 6436 8545
> > Fax: +61 8 6436 8555
> > Email: industry at ivec.org
> > WWW:  http://www.ivec.org
> >
> >
> >
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org To change your subscription
> > (digest mode or unsubscribe) visit
> > http://www.beowulf.org/mailman/listinfo/beowulf
> >
> >
> >
> --
> Keith D. Underwood                            Scalable Computing
> Systems
> Senior Member of Technical Staff            Sandia National
> Laboratories
> kdunder at sandia.gov
> 
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org To change your subscription
> (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf