[Beowulf] InfiniBand channel bundling?

[Peter Kjellström]

On Fri, 31 Oct 2014 01:17:49 -0400 (EDT)
Mark Hahn <hahn at mcmaster.ca> wrote:

> > If you compare QDR devices to FDR devices, than FDR is showing lower
> >latency.
> 
> in the paper referenced, that is not the case.  the numbers provided
> are QDR 1.27 us, versus FDR 1.67 us.  although it's only 400ns,
> it's still >30% slower, when one might expect a speed improvement.

These numbers match almost perfectly with the numbers in my other
e-mail. That is, 1.27 is QDR on pre-sandy bridge systems and 1.67 is
the far socket on a sandy bridge.

If that's the case then the "slow FDR" observation is really the pci-e
suckiness over QPI on sandy bridge.

...
> that's interesting - do you mean that in order to achieve higher
> bandwidth, the error rate becomes a problem, necessitating RT/FEC?  I
> guess it's obvious from the shrinkage of allowed passive/copper cable
> lengths that SNR/BER is a big issue, but does this imply that going
> optical will reduce the latency cost for FDR?

Optical cables are not a magic fix-all. We have much better reliability
and lower BER on our FDR cupper vs our EoE.

/Peter