[Beowulf] NFS + IB?

[Bill Broadley]

I read through the beowulf archives for mentions of NFS + IB.  I found nothing
newer than 2012.

What are peoples current experience with NFS + IB?  I'm looking at the options
for smaller clusters with /home on NFS.  I'll leave distributed filesystems for
a separate discussion.

The two leading options seem to be IPoIB and RDMA.  Any comments on stability
and performance assuming a fairly recent kernel like 3.13 or newer?

I'd be especially interested in any particular settings like rsize, wsize, mtu
(2044 for datagram mode), connected mode vs datagram mode, and similar tuning
parameters.  If you tuned for random I/O or bandwidth please mention it.

I talked to one vendor that claims FDR has a larger than expected performance
advantage over QDR for NFS.  Can anyone verify or guess as to exactly why?

Not sure if the IPoIB tuning is relevant for NFS + RDMA.

I've read these links, if there are other useful links please let me know:
   https://www.kernel.org/doc/Documentation/filesystems/nfs/nfs-rdma.txt
   https://www.kernel.org/doc/Documentation/infiniband/ipoib.txt

http://www.mellanox.com/related-docs/prod_software/Performance_Tuning_Guide_for_Mellanox_Network_Adapters.pdf

I'm happy to summarise the results if people prefer sending info directly to me.