[Beowulf] What is rdma, ofed, verbs, psm etc?

[Lux, Jim (337C)]

RDMA - shared memory with a communications link - so it's not *really* a dual ported memory, but, rather, a simulation of it with a (virtual) memory space on each side, and some magic in the infrastructure that makes it seem like simultaneous access to the same physical memory.   I suppose there's really no need for physical memory to correspond to the virtual memory, at least not in a one-to-one mapping sense.

There are, of course, all kinds of concurrency issues and insuring that both sides see the same data - kind of ensuring per processor cache consistency on a multiprocessor machine.

 You'll also see the term reflective memory.

Jim Lux
(818)354-2075 (office)
(818)395-2714 (cell)

-----Original Message-----
From: Beowulf [mailto:beowulf-bounces at beowulf.org] On Behalf Of Christopher Samuel
Sent: Tuesday, September 19, 2017 4:54 PM
To: beowulf at beowulf.org
Subject: Re: [Beowulf] What is rdma, ofed, verbs, psm etc?

Great explanations Peter.

On 20/09/17 02:24, Peter Kjellström wrote:

> ofed: a software distribution of network drivers, libraries, utilities 
> typically used by users/applications to run on Infiniband (and other 
> networks supported by ofed).

To expand on that slightly, this also includes (to add to the acronym
soup) RoCE - RDMA over Converged Ethernet - in other words using Ethernet networks (with appropriate switches) to do the sort of RDMA that you can do over Infiniband.

This is important as unlike Infiniband you can't do RoCE out of the box with something like RHEL7 (at least in the experience of the folks I'm helping out here).

cheers,
Chris
-- 
 Christopher Samuel        Senior Systems Administrator
 Melbourne Bioinformatics - The University of Melbourne
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf