[Beowulf] Performance tuning for Jumbo Frames

Patrick Geoffray patrick at myri.com
Tue Dec 15 20:41:59 PST 2009

Bogdan Costescu wrote:
> long as it fits in one page. At another time, the switch was more
> likely to drop large frames under high load (maybe something to do
> with internal memory management), so the 9000bytes frames worked most
> of the time while the 1500bytes ones worked all the time...

This is an important point. The way hardware flow-control works in 
Ethernet, a switch has to be able to buffer two full frames plus the 
time on the wire for the round-trip. For the curious, the PAUSEs packets 
are sent in-band and you cannot send or receive partial frames.
So, instead of requiring ~4K per port minimum, you need about ~20K per 
port. Add to that up to 8 priorities with DCB and the buffering 
requirement are quickly getting out of hand. That's one big drawbacks of 
large MTUs, along with contention with wormhole switching.


More information about the Beowulf mailing list