[Beowulf] bandwidth: who needs it?
lindahl at pathscale.com
Thu Oct 21 10:58:55 PDT 2004
> do you have applications that are pushing the limits of MPI bandwidth?
> for instance, code that actually comes close to using the 8-900 MB/s
> that current high-end interconnect provides?
Bandwidth is important not only for huge messages that hit 900 MB/s,
but also for medium sized messages. A naive formula for how long it
takes to send a message is:
T_size = T_0 + size / max_bandwidth
For example, for a 4k message with T_0 = 5 usec and either 400 MB/s or
T_4k_400M = 5 + 4k/400M = 5 + 10 = 15 usec
T_4k_800M = 5 + 4k/800M = 5 + 5 = 10 usec
A big difference. But you're only getting 266 MB/s and 400 MB/s
Of course performance is usually a bit less than this naive model. But
the effect is real, becoming unimportant for packets smaller than ~ 2k
in this example. The size at which this effect becomes unimportant
depends on T_0 and the bandwidth.
More information about the Beowulf