[Beowulf] how fast can NFS run?

[Dan Stromberg]

Oh, also, on the subject of NFS performance tuning:

http://dcs.nac.uci.edu/~strombrg/nfs-test.html

It's a script for empirically finding good rsize and wsize for a given
link.  Also can vary TCP and UDP.

BTW, it's sometimes useful to use different values for rsize and wsize.

On Tue, 2006-01-31 at 23:11 -0700, ctierney at hypermall.net wrote:
> Quoting Bruce Allen <ballen at gravity.phys.uwm.edu>:
> 
> > I'd like to know the fastest that anyone has seen an NFS server run, over 
> > either a 10Gb/s ethernet link or a handful of link aggregated 
> > (channel-bonded) Gb/s ethernet lines.
> > 
> > This would be with a small number of clients making large file sequential 
> > reads from the same NFS host/server.  Please assume that the NFS server 
> > has 'infinitely fast' disks.
> > 
> > I am told by one vendor that "NFS can't run faster than 100MB/sec".  I 
> > don't understand or believe this.  If the server's local disks can 
> > read/write at 300MB/s and the networking can run substantially faster than 
> > 100 MB/s, I don't see any constraint to faster operation.  But perhaps 
> > someone on this list can provide real-world data (or say why it can't 
> > work).
> 
> I built some servers a couple of years ago and was able to run
> NFS at about 100 MB/s.  The system was Dual socket Xeon, single gigE link, 1
> Gb/s fibre channel, XFS filesystem, jumbo frames.  The limiting factor was that
> the local filesystem could only do 100 MB/s.  Jumbo frames increased the
> performance from 60 MB/s to about 95 MB/s.
> 
> I know those servers could have gone faster if I had another gigE link
> and faster disk.  Going above 100 MB/s shouldn't be a problem.
> 
> If you want some other evidence, go look at www.agami.com.  They are a
> relatively new storage applicance company.  They claim their servers can
> provide
> up to 1000 MB/s over NFS (quad socket opteron, linux+xfs, lots of disks).
> I don't know if they tweaked the kernel or NFS to get this performance, but
> from the hardware they use I believe they could get that performance.  
> It certainly should be able to sustain more than 100 MB/s.
> 
> I am not endorsing the product, I am just pointing out some data that may
> be able to help you to determine what you can/should get out of a Linux Based
> NFS server.
> 
> Craig
> 
> 
> 
> > 
> > Note: I am free to use modern versions of the NFS protocol, jumbo frames, 
> > large rsize/wsize, etc.
> > 
> > Cheers,
> >  	Bruce
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org
> > To change your subscription (digest mode or unsubscribe) visit
> > http://www.beowulf.org/mailman/listinfo/beowulf
> > 
> 
> 
> 
> 
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>