NFS 100:1 performance loss

Mel Jones mello at
Fri Aug 18 15:49:48 PDT 2000

Josip Loncaric wrote:

> Has anyone seen this NFS performance problem?
> Our Beowulf has two servers w/RAID-0 arrays which deliver 45MB/s,
> connected via Gigabit Ethernet.  The machines are dual PIII/500 systems
> with 512MB of RAM each.  We are running Red Hat 6.2 updated to kernel
> 2.2.16-3 and recently we updated to nfs-utils-  We are running
> 16 kernel nfsd threads on each machine.  Here is how long it takes to
> copy a 28,955,860 byte file from machine 1 to machine 2:
> rcp:       1.04 seconds (27.8 MB/s, where 1MB=10^6B)
> ftp:       1.12 seconds (25.8 MB/s)
> NFS 1KB:  12.35 seconds ( 2.3 MB/s, using rsize=wsize=1024)
> NFS 8KB: 129.42 seconds ( 0.2 MB/s, using rsize=wsize=8192)
> Clearly, there is something very wrong with NFS, particularly with 8KB
> rsize/wsize (which should have improved performance!).  Our system
> manager tells me that reduced Linux NFS performance with 8KB rsize/wsize
> is a known problem, but even at 1KB our NFS is getting less than 10% of
> the rcp or ftp performance.
> Any ideas?
> Josip
> --
> Dr. Josip Loncaric, Senior Staff Scientist        mailto:josip at
> ICASE, Mail Stop 132C           PGP key at
> NASA Langley Research Center             mailto:j.loncaric at
> Hampton, VA 23681-2199, USA    Tel. +1 757 864-2192  Fax +1 757 864-6134
> _______________________________________________
> Beowulf mailing list
> Beowulf at

I've seen a similar but nowhere near as bad result on our network. This may
be completely irrelevant but I found NFS traffic between my SGI machines
about 4x slower than FTP. It got much better when I set the transport to
TCP rather than UDP.  The problem turned out to be that my switches weren't
necessarily forwarding the packets in order of receipt and this was taken
care of much more efficiently on the receiving machine with TCP.

            ... mello

More information about the Beowulf mailing list