[Beowulf] What are people seeing performance-wise for NFS over 10GbE
landman at scalableinformatics.com
Thu Feb 28 05:53:36 PST 2008
Have a few simple setups that I am trying to figure out if I have a
problem, or if what I am seeing is normal.
Two 10 GbE cards, connected with a CX4 cable. Server is one of our
JackRabbits, with 750+ MB/s direct IO and 500-650 MB/s buffered IO
(read-write), for IO about 10x system ram (~100x RAID cache).
Getting ok iperf numbers, about 7 Gb/s single thread. Running NFS,
and seeing ~200-300 MB/s best case between the systems. Is this what
others have seen? Latest drivers from vendor, sent a note to them to
see if we can figure this out. Figured I would tap into the 'wulf
collective memory to learn what other people see.
18.104.22.168 kernel on both sides, jumbo frames enabled. No switch, just
a CX4 cable.
My expectations are that we would be able to see 80-90% of the
JackRabbits speed, with the rest being eaten by stack issues. This is
the case over channel bonded GbE.
Do you see 500 MB/s or more? Less?
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax : +1 866 888 3112
cell : +1 734 612 4615
More information about the Beowulf