[Beowulf] InfiniBand channel bundling?

[Gilad Shainer]

If you compare QDR devices to FDR devices, than FDR is showing lower latency. What you might heard is that the FDR switches are slightly higher latency than the QDR switches as they include new capabilities of link level retransmission and forward error correction, but overall end to end latency with FDR is lower. The EDR switch latency is lower than the FDR switch and the QDR switch, so further latency decrease will be seen with EDR. 

Regards,
Gilad Shainer 
 


-----Original Message-----
From: Beowulf [mailto:beowulf-bounces at beowulf.org] On Behalf Of Prentice Bisbal
Sent: Thursday, October 30, 2014 7:49 AM
To: beowulf at beowulf.org
Subject: Re: [Beowulf] InfiniBand channel bundling?

Gilad,

If end-to-end is lower for FDR, then what latency is being measured for FDR that is higher than for QDR? According to Wikipedia, and the Mellanox website, FDR does have a better latency than QDR (0.7 microseconds vs. 1.3 microseconds), but I and others on this list have heard that FDR has worse latency than QDR. Have we been misinformed, or does it depend on how you measure or define latency?

http://en.wikipedia.org/wiki/InfiniBand
http://www.mellanox.com/page/performance_infiniband

Prentice


On 10/29/2014 06:46 PM, Gilad Shainer wrote:
> End-to-end FDR latency is lower than end-to-end QDR latency - per published measurments that can be found in multiple places.
>
> Gilad
>
> -----Original Message-----
> From: Beowulf [mailto:beowulf-bounces at beowulf.org] On Behalf Of Jörg 
> Saßmannshausen
> Sent: Wednesday, October 29, 2014 3:45 PM
> To: Beowulf Mailinglist
> Subject: Re: [Beowulf] InfiniBand channel bundling?
>
> Hi all,
>
> thanks again for the wealth of information.
>
> Now, given that I am not interested in transporting files over the IB network but I am doing parallel calculations, I would have thought that the latency here is more important than the speed?
> Thus, if FDR has a higher latency than QDR, does that mean my performance is decreasing when I am running a calculation between nodes?
>
> For those of you who are into Chemistry code: I am using VASP, cp2k, quantum espresso and cpmd mainly. All of that is plain wave code.
>
> All the best from a wet London
>
> Jörg
>
>
> On Mittwoch 29 Oktober 2014 Prentice Bisbal wrote:
>> On 10/28/2014 04:43 PM, Mark Hahn wrote:
>>> On Tue, 28 Oct 2014, John Hearns wrote:
>>>> Here is a very good post from Glenn Lockwood regarding FDR versus 
>>>> dual-rail QDR:
>>>>
>>>> http://glennklockwood.blogspot.co.uk/2013/05/fdr-infiniband-vs-dual
>>>> -rail
>>>> -qdr.html
>>> indeed, very nice.  though also quite surprising - is it known that 
>>> FDR is so terrible for latency?  seems astonishing to me.
>> Yes, it was known to me. I had already known that FDR was worse than 
>> QDR for latency, but I don't remember my source. I don't know if I'd 
>> characterize it as "so terrible", though.
>
> --
> *************************************************************
> Dr. Jörg Saßmannshausen, MRSC
> University College London
> Department of Chemistry
> Gordon Street
> London
> WC1H 0AJ
>
> email: j.sassmannshausen at ucl.ac.uk
> web: http://sassy.formativ.net
>
> Please avoid sending me Word or PowerPoint attachments.
> See http://www.gnu.org/philosophy/no-word-attachments.html
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin 
> Computing To change your subscription (digest mode or unsubscribe) 
> visit http://www.beowulf.org/mailman/listinfo/beowulf

--
Prentice Bisbal
Manager of Information Technology
Rutgers Discovery Informatics Institute (RDI2) Rutgers University http://rdi2.rutgers.edu

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf