thanks Ryan, I forgot about smpquery. while the info is still
cumbersome, its easier to parse then iblinkinfo
# smpquery -D nodeinfo 0
# Node info: DR path slid 65535; dlid 65535; 0
BaseVers:........................1
ClassVers:.......................1
NodeType:........................Channel Adapter
NumPorts:........................1
SystemGuid:......................0x88e9a4ffff404704
Guid:............................0x88e9a4ffff404704
PortGuid:........................0x88e9a4ffff404704
PartCap:.........................128
DevId:...........................0x101b
Revision:........................0x00000000
LocalPort:.......................1
VendorId:........................0x0002c9
# smpquery -D nodeinfo 0,1
# Node info: DR path slid 65535; dlid 65535; 0,1
BaseVers:........................1
ClassVers:.......................1
NodeType:........................Switch
NumPorts:........................36
SystemGuid:......................0x080069000000fbcf
Guid:............................0x080069000000fbcc
PortGuid:........................0x080069000000fbcc
PartCap:.........................8
DevId:...........................0xc738
Revision:........................0x000000a1
LocalPort:.......................29
VendorId:........................0x0002c9
# smpquery -D nodedesc 0,1
Node Description:.SwitchX - Mellanox Technologies
On Wed, Sep 20, 2023 at 4:01 PM Ryan Novosielski <novosirj at rutgers.edu> wrote:
>
> Yeah, that’s the analogy I thought of after I asked.
>
> Have you checked smpquery? I’m not 100% sure that that’s it, but I think it was one of those commands that doesn’t start with ib* that could do this.
>
> --
> #BlackLivesMatter
> ____
> || \\UTGERS, |---------------------------*O*---------------------------
> ||_// the State | Ryan Novosielski - novosirj at rutgers.edu
> || \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
> || \\ of NJ | Office of Advanced Research Computing - MSB A555B, Newark
> `'
>
> On Sep 20, 2023, at 14:49, Michael DiDomenico <mdidomenico4 at gmail.com> wrote:
>
> meaning, when on a server, the CA port is connected to either a switch
> or another CA (back-to-back/netapp storage). i just need to see whats
> on the other side of the cable from the server i'm sitting on. i
> don't care about the rest of the fabric
>
> its analogous to asking what switch/port i'm connected to via cdp/lldp
> on ethernet
>
> On Wed, Sep 20, 2023 at 2:38 PM Ryan Novosielski <novosirj at rutgers.edu> wrote:
>
>
> Neighbor means what here, just one port over? Or the machine at the other end of the port, or…?
>
> Sent from my iPhone
>
> On Sep 20, 2023, at 14:22, Michael DiDomenico <mdidomenico4 at gmail.com> wrote:
>
> thanks for the info. ibnetdiscover doesn't give me anymore then
> iblinkinfo does (and in just as bad a format), however, running
> ibnetdiscover on a network with +15k machines is unpleasant (so is
> iblinkinfo, but at least i can restrict the hops). especially since
> all i need to know is who my neighbor is on a compute node
>
> netloc looks interesting, it doesn't seem to be included with the
> rhel8 version of hwloc though. despite being v2.2, which according to
> openmpi says it's included by default. i'll have to look into it
>
> i did manage to hack up a python program to pull the data into form,
> its ugly, but it'll do for now. i'm sure this easily done with a
> short C program, but my C programming skills aren't up to the task.
>
>
> On Wed, Sep 20, 2023 at 9:07 AM John Hearns <hearnsj at gmail.com> wrote:
>
> netloc is the tool you want to use.
> Look in the latest hwloc dovumentation
>
> On Wed, 20 Sep 2023, 13:55 John Hearns, <hearnsj at gmail.com> wrote:
>
> I did manage to get the graphical netloc utility working once. Part of the hwloc/openmpi project.
>
> It produces a very pretty image of I topology. I think if you zoom in you can get neighbours.
> A few years since I used it.
>
> On Tue, 19 Sep 2023, 19:03 Michael DiDomenico, <mdidomenico4 at gmail.com> wrote:
>
>
> does anyone know if there's a simple command to pull the neighbor of
> the an ib port? for instance, this horrible shell command line
>
> # for x in `ibstat | awk -F \' '/^CA/{print $2}'`; do iblinkinfo -C
> ${x} -n 1 -l | grep `hostname -s`; done
> 0x080069000000fbcc "SwitchX - Mellanox Technologies" 41134 29[ ]
> ==( 4X 14.0625 Gbps Active/ LinkUp)==> 0x88e9a4ffff404704
> 611 1[ ] "logs01 HCA-1" ( )
> 0x88e9a4ffff404704 " logs01 HCA-1" 611 1[ ]
> ==( 4X 14.0625 Gbps Active/ LinkUp)==> 0x080069000000fbcc
> 41134 29[ ] "SwitchX - Mellanox Technologies" ( )
> 0x080069000000fbdc "SwitchX - Mellanox Technologies" 41219 29[ ]
> ==( 4X 14.0625 Gbps Active/ LinkUp)==> 0x88e9a4ffff404705
> 10105 1[ ] "logs01 HCA-2" ( )
> 0x88e9a4ffff404705 " logs01 HCA-2" 10105 1[ ]
> ==( 4X 14.0625 Gbps Active/ LinkUp)==> 0x080069000000fbdc
> 41219 29[ ] "SwitchX - Mellanox Technologies" ( )
>
> outputs what i need (though i only need the CA perspective), but it's
> going to be an atrocious effort in text parsing. would be nice if
> there was a nice simple command, preferably that outputs json, but
> that's likely wishful thinking
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
>
>