[Beowulf] MPI + IB question

[Bogdan Costescu]

On Thu, Nov 15, 2012 at 2:10 PM, Jörg Saßmannshausen
<j.sassmannshausen at ucl.ac.uk> wrote:
> thanks for the feedback. I actually done a test of the IB-network after
> installation (the typical Ping-Pong) with the results attached to this email.
>
> Also, a different program (cp2k) is running without any problems when I am
> using the IB network (as enforced with the  --mca btl ^tcp).

Your previous results have shown a particular pair of nodes not able
to communicate via IB. Have you tested this particular pair ? Also
tests between each of these nodes and another node ?

> One thing came to mind: when I am disabling the TCP network, does that mean
> that the lifeline will be also over the IB network or does OpenMPI still uses
> the TCP network to connect to the other nodes and then uses the IB network for
> the communication?

OpenMPI always uses TCP for the inter-node lifeline, from what I know.
The selected BTLs are used to move data bytes around, but the lifeline
is not considered as moving data. The messages you've shown come from
the BTL selection, the lifeline is not involved here.

Cheers,
Bogdan