[Beowulf] InfiniBand channel bundling?

[Jörg Saßmannshausen]

Dear all,

thanks for the feedback.

I think the best thing to do is to run a few test jobs and see what happens 
here. Once that is done I can report back so the community gets something back 
from me as well. 

I did not expect it to be that easy to be honest. I know that OpenMPI does 
test the various interfaces but I did not expect that it is using the two IB 
ports if they are available. 

So thanks for your help so far!

All the best from London

Jörg


On Montag 27 Oktober 2014 Prentice Bisbal wrote:
> On 10/27/2014 12:03 PM, Kilian Cavalotti wrote:
> > Hi Jörg,
> > 
> > On Mon, Oct 27, 2014 at 3:56 AM, Jörg Saßmannshausen
> > 
> > <j.sassmannshausen at ucl.ac.uk> wrote:
> >> I got some older dual Mellanox Technologies MT23108 cards (8.5 Gb/sec
> >> (4X)) and currently I am only using one of the two ports on them.
> >> I was wondering, is it possible with InfiniBand to utilise the second
> >> port in such a way that I can increase the bandwith of the network (and
> >> lower the latency maybe)?
> > 
> > Last time I checked, the Linux bonding driver didn't allow to bond IB
> > interfaces to increase throughput. I'd be happy to hear that this
> > changed, but I think only redundancy modes (such as active-backup)
> > were possible.
> > 
> > Now, Open MPI can take advantage of multiple, independent ports
> > connected to the same fabric: see
> > http://www.open-mpi.org/faq/?category=openfabrics#ofa-port-wireup for
> > details.
> > 
> > Cheers,
> 
> This makes sense. Since IB operates mostly in userspace, I would expect
> setting up IB like this to done by the applications, not the operating
> system. Since TCP/IP network is mostly handled by the kernel, it would
> make sense that IPoIB would be configured by the kernel, though. Feel
> free to correct me if this logic is wrong.
> 
> You could increase the bandwidth, but not the latency. If anything, the
> latency might go up as some additional work will need to be done to
> coordinate the data going over the two different connections.


-- 
*************************************************************
Dr. Jörg Saßmannshausen, MRSC
University College London
Department of Chemistry
Gordon Street
London
WC1H 0AJ 

email: j.sassmannshausen at ucl.ac.uk
web: http://sassy.formativ.net

Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 230 bytes
Desc: This is a digitally signed message part.
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20141027/8e153a53/attachment.sig>