NFS 100:1 performance loss

[Mel Jones]

Josip Loncaric wrote:

> Has anyone seen this NFS performance problem?
>
> Our Beowulf has two servers w/RAID-0 arrays which deliver 45MB/s,
> connected via Gigabit Ethernet.  The machines are dual PIII/500 systems
> with 512MB of RAM each.  We are running Red Hat 6.2 updated to kernel
> 2.2.16-3 and recently we updated to nfs-utils-0.1.9.1-1.  We are running
> 16 kernel nfsd threads on each machine.  Here is how long it takes to
> copy a 28,955,860 byte file from machine 1 to machine 2:
>
> rcp:       1.04 seconds (27.8 MB/s, where 1MB=10^6B)
> ftp:       1.12 seconds (25.8 MB/s)
> NFS 1KB:  12.35 seconds ( 2.3 MB/s, using rsize=wsize=1024)
> NFS 8KB: 129.42 seconds ( 0.2 MB/s, using rsize=wsize=8192)
>
> Clearly, there is something very wrong with NFS, particularly with 8KB
> rsize/wsize (which should have improved performance!).  Our system
> manager tells me that reduced Linux NFS performance with 8KB rsize/wsize
> is a known problem, but even at 1KB our NFS is getting less than 10% of
> the rcp or ftp performance.
>
> Any ideas?
> Josip
>
> --
> Dr. Josip Loncaric, Senior Staff Scientist        mailto:josip at icase.edu
> ICASE, Mail Stop 132C           PGP key at http://www.icase.edu./~josip/
> NASA Langley Research Center             mailto:j.loncaric at larc.nasa.gov
> Hampton, VA 23681-2199, USA    Tel. +1 757 864-2192  Fax +1 757 864-6134
>
> _______________________________________________
> Beowulf mailing list
> Beowulf at beowulf.org
> http://www.beowulf.org/mailman/listinfo/beowulf

I've seen a similar but nowhere near as bad result on our network. This may
be completely irrelevant but I found NFS traffic between my SGI machines
about 4x slower than FTP. It got much better when I set the transport to
TCP rather than UDP.  The problem turned out to be that my switches weren't
necessarily forwarding the packets in order of receipt and this was taken
care of much more efficiently on the receiving machine with TCP.

            ... mello