[Beowulf] NFS over RDMA
landman at scalableinformatics.com
Fri Oct 24 10:11:22 PDT 2008
John Hearns wrote:
> Joe has just mentioned NFS over RDMA, a subject which I have somewhat of
> a passing interest in.
> What's the perceived wisdom on it? Biggest, best shiny new thing on the
> block? Works just fine?
> Or should it be in the basement inside a locked filing cabinet in a
> disused lavatory with a sign saying Beware of the Leopard?
FWIW: we see ~460 MB/s over SDR pairs, going disk to disk (that is real
disk on JackRabbit behind the RDMA as opposed to ram disk or null
devices which give obscenely fast, and practically unattainable numbers
in real world scenarios).
We have wedged one machine running it, requiring a LART (reset button
press) on the client side. On the server side, it still is finicky but
this is mostly on initial mount.
Requires mount-utils of 1.1.1 or higher (1.1.2 or higher is
recommended). Requires kernels of 2.6.25 or higher on both sides due to
in kernel support. Not sure if the OFED stack does some sort of magic
here for people.
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web : http://www.scalableinformatics.com
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
More information about the Beowulf