[Beowulf] ib neighbor

[Ryan Novosielski]

Yeah, that’s the analogy I thought of after I asked.

Have you checked smpquery? I’m not 100% sure that that’s it, but I think it was one of those commands that doesn’t start with ib* that could do this.

--
#BlackLivesMatter
____
|| \\UTGERS,     |---------------------------*O*---------------------------
||_// the State  |         Ryan Novosielski - novosirj at rutgers.edu
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ  | Office of Advanced Research Computing - MSB A555B, Newark
     `'

On Sep 20, 2023, at 14:49, Michael DiDomenico <mdidomenico4 at gmail.com> wrote:

meaning, when on a server, the CA port is connected to either a switch
or another CA (back-to-back/netapp storage).  i just need to see whats
on the other side of the cable from the server i'm sitting on.  i
don't care about the rest of the fabric

its analogous to asking what switch/port i'm connected to via cdp/lldp
on ethernet

On Wed, Sep 20, 2023 at 2:38 PM Ryan Novosielski <novosirj at rutgers.edu> wrote:

Neighbor means what here, just one port over? Or the machine at the other end of the port, or…?

Sent from my iPhone

On Sep 20, 2023, at 14:22, Michael DiDomenico <mdidomenico4 at gmail.com> wrote:

thanks for the info.  ibnetdiscover doesn't give me anymore then
iblinkinfo does (and in just as bad a format), however, running
ibnetdiscover on a network with +15k machines is unpleasant (so is
iblinkinfo, but at least i can restrict the hops).  especially since
all i need to know is who my neighbor is on a compute node

netloc looks interesting, it doesn't seem to be included with the
rhel8 version of hwloc though.  despite being v2.2, which according to
openmpi says it's included by default.  i'll have to look into it

i did manage to hack up a python program to pull the data into form,
its ugly, but it'll do for now.  i'm sure this easily done with a
short C program, but my C programming skills aren't up to the task.


On Wed, Sep 20, 2023 at 9:07 AM John Hearns <hearnsj at gmail.com> wrote:

netloc is the tool you want to use.
Look in the latest hwloc dovumentation

On Wed, 20 Sep 2023, 13:55 John Hearns, <hearnsj at gmail.com> wrote:

I did manage to get the graphical netloc utility working once. Part of the hwloc/openmpi project.

It produces a very pretty image of I topology. I think if you zoom in you can get neighbours.
A few years since I used it.

On Tue, 19 Sep 2023, 19:03 Michael DiDomenico, <mdidomenico4 at gmail.com> wrote:

does anyone know if there's a simple command to pull the neighbor of
the an ib port?  for instance, this horrible shell command line

# for x in `ibstat | awk -F \' '/^CA/{print $2}'`; do iblinkinfo -C
${x} -n 1 -l | grep `hostname -s`; done
0x080069000000fbcc "SwitchX -  Mellanox Technologies"  41134   29[  ]
==( 4X       14.0625 Gbps Active/  LinkUp)==>  0x88e9a4ffff404704
611    1[  ] "logs01 HCA-1" ( )
0x88e9a4ffff404704 "                  logs01 HCA-1"    611    1[  ]
==( 4X       14.0625 Gbps Active/  LinkUp)==>  0x080069000000fbcc
41134   29[  ] "SwitchX -  Mellanox Technologies" ( )
0x080069000000fbdc "SwitchX -  Mellanox Technologies"  41219   29[  ]
==( 4X       14.0625 Gbps Active/  LinkUp)==>  0x88e9a4ffff404705
10105    1[  ] "logs01 HCA-2" ( )
0x88e9a4ffff404705 "                  logs01 HCA-2"  10105    1[  ]
==( 4X       14.0625 Gbps Active/  LinkUp)==>  0x080069000000fbdc
41219   29[  ] "SwitchX -  Mellanox Technologies" ( )

outputs what i need (though i only need the CA perspective), but it's
going to be an atrocious effort in text parsing.   would be nice if
there was a nice simple command, preferably that outputs json, but
that's likely wishful thinking
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://beowulf.org/pipermail/beowulf/attachments/20230920/5254f51e/attachment.htm>