[Beowulf] InfiniBand channel bundling?

[Brice Goglin]

Do you want to compare FDR vs QDR when used in the exact same PCIe
slots? FDR often goes in Gen3 while QDR has been in Gen2 for a while.

According to the link below, Gen3 is responsible for pretty much all the
improvement between QDR/Gen2 and FDR/Gen3.
http://en.community.dell.com/techcenter/high-performance-computing/b/general_hpc/archive/2012/03/06/hpc-i-o-performance-using-pci-e-gen3-slots-on-the-12th-generation-12g-poweredge-servers

Brice



Le 30/10/2014 15:41, Prentice Bisbal a écrit :
> Gilad,
>
> If end-to-end is lower for FDR, then what latency is being measured
> for FDR that is higher than for QDR? According to Wikipedia, and the
> Mellanox website, FDR does have a better latency than QDR (0.7
> microseconds vs. 1.3 microseconds), but I and others on this list have
> heard that FDR has worse latency than QDR. Have we been misinformed,
> or does it depend on how you measure or define latency?
>
> http://en.wikipedia.org/wiki/InfiniBand
> http://www.mellanox.com/page/performance_infiniband
>
> Prentice
>
>
> On 10/29/2014 06:46 PM, Gilad Shainer wrote:
>> End-to-end FDR latency is lower than end-to-end QDR latency - per
>> published measurments that can be found in multiple places.
>>
>> Gilad
>>
>> -----Original Message-----
>> From: Beowulf [mailto:beowulf-bounces at beowulf.org] On Behalf Of Jörg
>> Saßmannshausen
>> Sent: Wednesday, October 29, 2014 3:45 PM
>> To: Beowulf Mailinglist
>> Subject: Re: [Beowulf] InfiniBand channel bundling?
>>
>> Hi all,
>>
>> thanks again for the wealth of information.
>>
>> Now, given that I am not interested in transporting files over the IB
>> network but I am doing parallel calculations, I would have thought
>> that the latency here is more important than the speed?
>> Thus, if FDR has a higher latency than QDR, does that mean my
>> performance is decreasing when I am running a calculation between nodes?
>>
>> For those of you who are into Chemistry code: I am using VASP, cp2k,
>> quantum espresso and cpmd mainly. All of that is plain wave code.
>>
>> All the best from a wet London
>>
>> Jörg
>>
>>
>> On Mittwoch 29 Oktober 2014 Prentice Bisbal wrote:
>>> On 10/28/2014 04:43 PM, Mark Hahn wrote:
>>>> On Tue, 28 Oct 2014, John Hearns wrote:
>>>>> Here is a very good post from Glenn Lockwood regarding FDR versus
>>>>> dual-rail QDR:
>>>>>
>>>>> http://glennklockwood.blogspot.co.uk/2013/05/fdr-infiniband-vs-dual
>>>>> -rail
>>>>> -qdr.html
>>>> indeed, very nice.  though also quite surprising - is it known that
>>>> FDR is so terrible for latency?  seems astonishing to me.
>>> Yes, it was known to me. I had already known that FDR was worse than
>>> QDR for latency, but I don't remember my source. I don't know if I'd
>>> characterize it as "so terrible", though.
>>
>> -- 
>> *************************************************************
>> Dr. Jörg Saßmannshausen, MRSC
>> University College London
>> Department of Chemistry
>> Gordon Street
>> London
>> WC1H 0AJ
>>
>> email: j.sassmannshausen at ucl.ac.uk
>> web: http://sassy.formativ.net
>>
>> Please avoid sending me Word or PowerPoint attachments.
>> See http://www.gnu.org/philosophy/no-word-attachments.html
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>