[Beowulf] ipoib routing

[Adam DeConinck]

Hi Michael,

This sounds a lot like the "I/O node" arrangement which was used when
I was at LANL. I/O nodes attached to each cluster's IB network were
also attached to the "external" ethernet network, and provided access
to shared filesystems and other resources. The I/O nodes routed
between the two using OSPF, I think using the quagga daemon.

There's a presentation from one of our network folks that gives some
detail here: https://permalink.lanl.gov/object/tr?what=info:lanl-repo/lareport/LA-UR-16-28013

There's not much in the way of config detail there, but I can probably
get you in touch with the right folks off-list if needed.

Cheers,
Adam

On Tue, Dec 11, 2018 at 8:03 AM Michael Di Domenico
<mdidomenico4 at gmail.com> wrote:
>
> does anyone use linux based routers to get from an ipoib network out
> onto a regular ethernet network?  if so, do you have multiple routers
> or just one?  in the case of multiple routers how to you handle the
> traffic coming back from the ethernet network?  (ie how does the
> ethernet network know which router to send the data back through)
>
> ---
>
> to expand further on what i'm trying to do (sanity/success is still to
> be determine)...
>
> imagine a big cluster of machines all connected via infiniband running ipoib
> there are 4 linux routers on the ipoib network that also have 10Gb
> connections to an ethernet network
> the ethernet network contains a router to other resources
>
> what i want to accomplish is to have the compute nodes spray their
> traffic across the 4 linux routers and end up on the ethernet network
> i'm sure someone's done something like this before, but my google foo
> is failing me
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf