[Beowulf] mixing of Infiniband HCAs in cluster

[Gil Bloch]

> 
> We have a pair of IB 4x (Mellanox MTLP23108) HCAs used currently for
> direct switchless connection of pair of nodes.
> 
> Now we plan to buy 8-port (MTK37900) or 24-port IB switch w/HCAs,
> which may be also DDR IB. And we want to build this cluster including
> also old pair of nodes w/MTLP23108 HCAs. But is this possible - to
> have both HCA kinds (DDR and usual IB 4x HCAs) connected to the same
> switch ? If the answer is "yes", is it necessary to setup all the
> DDR-HCA drivers manually for work w/"4x speed" or all the necessary
> things will be done via negotiations ?
> 

InfiniBand defines auto-negotiation of link speed, and each of the
switch port will negotiate speed independently of other port. All ports
configuration will be done automatically.

The only thing you should notice is that streams from DDR HCAs to SDR
HCAs should reduce data rate to avoid flooding the switches. This is
done on a per-stream (QP) base, thus the DDR port can be fully utilized.
Using CMA for connection settings will take care of it automatically. If
your application does not use CMA, you should take care of it somehow
(e.g. use CMA).

> Yours
> Mikhail Kuzminsky
> Computer Assistance to Chemical Research Center,
> Zelinsky Institute of Organic Chemistry
> Moscow
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>