[Beowulf] InfiniBand channel bundling?

[Mark Hahn]

>      bw      lat(near cpu)   lat(far cpu)
> --------------------------------------------------
> QDR   3300*   1.12            1.54
> FDR   6000    0.85            1.59

thanks for the numbers, though they are somewhat perplexing.

> QDR hw is mlx connectx3 to mlx switchX (same switch), ivy bridge node.
> FDR hw is mlx connectx3 to mlx switchX (same switch), sandy bridge node.

hmm, so do you have an sense for what differences are due to
host differences?  also, what does near/far mean if this is all
on the same switch?  (simple 36pt switch, or is it a big switch
with an internal fat tree?)

> Of course what people refer to as QDR could also be connectx2 or
> qlogic/intel. The smaller hit for "far cpu" for qdr could in my case be
> an improvment from snb to ivb.

that kinda makes it sound like near/far is just the in-node topology
(far means +1 QPI hop?)

thanks, mark hahn.