NFS 100:1 performance loss

[Josip Loncaric]

Has anyone seen this NFS performance problem?

Our Beowulf has two servers w/RAID-0 arrays which deliver 45MB/s,
connected via Gigabit Ethernet.  The machines are dual PIII/500 systems
with 512MB of RAM each.  We are running Red Hat 6.2 updated to kernel
2.2.16-3 and recently we updated to nfs-utils-0.1.9.1-1.  We are running
16 kernel nfsd threads on each machine.  Here is how long it takes to
copy a 28,955,860 byte file from machine 1 to machine 2:

rcp:       1.04 seconds (27.8 MB/s, where 1MB=10^6B)
ftp:       1.12 seconds (25.8 MB/s)
NFS 1KB:  12.35 seconds ( 2.3 MB/s, using rsize=wsize=1024)
NFS 8KB: 129.42 seconds ( 0.2 MB/s, using rsize=wsize=8192)

Clearly, there is something very wrong with NFS, particularly with 8KB
rsize/wsize (which should have improved performance!).  Our system
manager tells me that reduced Linux NFS performance with 8KB rsize/wsize
is a known problem, but even at 1KB our NFS is getting less than 10% of
the rcp or ftp performance.

Any ideas?
Josip


-- 
Dr. Josip Loncaric, Senior Staff Scientist        mailto:josip at icase.edu
ICASE, Mail Stop 132C           PGP key at http://www.icase.edu./~josip/
NASA Langley Research Center             mailto:j.loncaric at larc.nasa.gov
Hampton, VA 23681-2199, USA    Tel. +1 757 864-2192  Fax +1 757 864-6134