[Beowulf] How to know if infiniband network works?

[John Hearns]

Fazar,
I think that you have got things sorted out.
However I think that the number of optiosn in OpenMPI is starting to
confuse you.  But do not lose heart!
I have been in the same place myself many time.  Specifically I am thinking
on one time when a customer asked me to benthmark the latency across 10Gbps
interfaces,
on a cluster where there was already a 1Gbps network and a Mellanox
Infiniband network. I had to be careful to exclude the networks I did NOT
want!

I suggest that you set the verbose flag in mpirun and keep a copy of the
output. GO through that output line by line making sure you understand what
it is telling you.
I have done that many times!


Secondly you say " I also tried changing the environment variable: export
OMPI_MCA_btl=tcp,self,sm "  - remember that you can switch OFF a transport
by using ^tcp
Please give that a try - ie I mean epxlicitly request openmpi transport and
^tcp








On 3 August 2017 at 16:21, Michael Di Domenico <mdidomenico4 at gmail.com>
wrote:

> On Thu, Aug 3, 2017 at 10:10 AM, Faraz Hussain <info at feacluster.com>
> wrote:
> > Thanks, I installed the MPI tests from Ohio State. I ran osu_bw and got
> the
> > results below. What is confusing is I get the same result if I use tcp or
> > openib ( by doing --mca btl openib|tcp,self with my mpirun command ). I
> also
> > tried changing the environment variable: export OMPI_MCA_btl=tcp,self,sm
> .
> > Results are the same regardless of tcp or openib..
> >
> > And when I do ifconfig -a I still see zero traffic reported for the ib0
> and
> > ib1 network.
>
> if openmpi uses RDMA for the traffic ib0/ib1 will not show traffic,
> you have to use perfquery
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20170803/c2866a67/attachment.html>