[Beowulf] Performance characterising a HPC application

[Gilad Shainer]

> -----Original Message-----
> From: beowulf-bounces at beowulf.org 
> [mailto:beowulf-bounces at beowulf.org] On Behalf Of Patrick Geoffray
> Sent: Thursday, March 22, 2007 4:43 AM
> To: beowulf at beowulf.org
> Subject: Re: [Beowulf] Performance characterising a HPC application
> 
> Greg Lindahl wrote:
> > On Wed, Mar 21, 2007 at 06:41:07AM -0400, Scott Atchley wrote:
> > 
> >> I have not benchmarked any applications that need more 
> than 250 MB/s 
> >> during computation,
> > 
> > There is a large class of computations which alternate 
> non-overlapped 
> > compute and communicate cycles. The average over the 
> lifetime of the 
> > process can be ~ 100 MB/s, but that could easily mean it is 
> sending at 
> > 1 GByte/s for 10% of the runtime. If you slow down 
> communications by a 
> > factor of 4, the app will run a lot slower.
> 
> In codes that alternate computation and communication (naive 
> but most common design), the communication phase is usually 
> much smaller that the computation, unless the problem is 
> imbalanced or too small. In my experience, 10% of the runtime 
> is realistic, but at 250 MB/s. So, when you bump the network 
> to 1 GB/s, you only gain 7.5% of runtime. That's why 1G 
> (GigE) is fine for a lot of cases, 2G is interesting for a 
> subset and 10G for even less apps.
> 
> As everybody is communicating at the same time, you need to 
> increase the network bandwidth when you increase the number 
> of cores, to keep constant resources per core. However, these 
> codes are often bounded by contention rather than bandwidth. 
> The communication pattern is all-to-all or, worse, a careless 
> hand-made exchange. Even with a clean all-to-all, slight 
> imbalances easily create contention in the network
> (N->1 contention), and the backpressure flow-control 
> propagates it everywhere as everybody is communicating at the 
> same time. Bigger pipes helps contention a bit, but not much.
> 
> People doing their homework are still buying more 2G than 10G 
> today, because of better price/performance for their codes 
> (and thin cables).
> 


People doing their homework test their applications and decide
accordingly. There are several centers that provide free run time
For that purpose. In your case, maybe your customers prefer your 2G
over your 10G, but I am really not sure if most of the HPC users
are buying more 2G rather than other faster I/O solutions..... 

All the real applications performance that I saw show that IB 10 
and 20Gb/s provide much higher performance results comparing to 
your 2G, and clearly better price/performance. This is a good 
indication that applications require more bandwidth than 2G.



> Patrick
> --
> Patrick Geoffray
> Myricom, Inc.
> http://www.myri.com
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org To change your 
> subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf
>