[Beowulf] NFS + IB?

[Orion Poplawski]

On 02/20/2015 05:15 PM, Bill Broadley wrote:
> I read through the beowulf archives for mentions of NFS + IB.  I found nothing
> newer than 2012.
>
> What are peoples current experience with NFS + IB?  I'm looking at the options
> for smaller clusters with /home on NFS.  I'll leave distributed filesystems for
> a separate discussion.
>
> The two leading options seem to be IPoIB and RDMA.  Any comments on stability
> and performance assuming a fairly recent kernel like 3.13 or newer?

I'm afraid I don't have anything on newer kernels, so I'd love to hear 
peoples experiences.  But with EL6.4 I had terrible stability issues 
with NFSoRDMA.  Starting nfs service in particular would lock up the 
machine hard.  So I have been sticking with IPoIB without much trouble. 
  Have not spent much time tuning/benchmarking.

https://bugzilla.redhat.com/show_bug.cgi?id=1031212 is an unfortunately 
private (kernel issues are automatically private now) bug about the nfs 
restart issue.  But I never got any response on it anyway.

-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA/CoRA Division                    FAX: 303-415-9702
3380 Mitchell Lane                  orion at cora.nwra.com
Boulder, CO 80301              http://www.cora.nwra.com