[Beowulf] InfiniBand channel bundling?
cap at nsc.liu.se
Mon Nov 3 01:36:46 PST 2014
On Fri, 31 Oct 2014 01:17:49 -0400 (EDT)
Mark Hahn <hahn at mcmaster.ca> wrote:
> > If you compare QDR devices to FDR devices, than FDR is showing lower
> in the paper referenced, that is not the case. the numbers provided
> are QDR 1.27 us, versus FDR 1.67 us. although it's only 400ns,
> it's still >30% slower, when one might expect a speed improvement.
These numbers match almost perfectly with the numbers in my other
e-mail. That is, 1.27 is QDR on pre-sandy bridge systems and 1.67 is
the far socket on a sandy bridge.
If that's the case then the "slow FDR" observation is really the pci-e
suckiness over QPI on sandy bridge.
> that's interesting - do you mean that in order to achieve higher
> bandwidth, the error rate becomes a problem, necessitating RT/FEC? I
> guess it's obvious from the shrinkage of allowed passive/copper cable
> lengths that SNR/BER is a big issue, but does this imply that going
> optical will reduce the latency cost for FDR?
Optical cables are not a magic fix-all. We have much better reliability
and lower BER on our FDR cupper vs our EoE.
More information about the Beowulf