[Beowulf] Network Filesystems performance

[Joe Landman]

Glen Dosey wrote:
> I am using the async flag. I'm also using the Infiniband now instead of
> the GigE and I've increased my default TCP window up to 6MB. None of it
> makes a difference. It's still around 40MB/s.

BTW:

 dstat, iftop, and atop are your friends.

http://dag.wieers.com/home-made/dstat/
http://dag.wieers.com/rpm/packages/iftop/
http://dag.wieers.com/rpm/packages/atop/

Nothing like doing an

	dstat -N total,eth0,eth1 -D sda,sdb,...

on both machines before starting the transfer, in different windows of
course ...

	iftop -B -i eth0

is known to be extraordinarily helpful at figuring out what bandwidth
you are really seeing.

Since you indicated RHEL4, its possible that something in kernel is
causing problems.  RHEL4 is not known to be a speed demon.

What about the usual suspects

	cat /proc/interrupts
	blockdev --getra /dev/sda
...
	lspci -v

Is your gigabit sharing a 100/133 MB/s old PCI bus with your RAID card?
On older motherboards, the gigabit NICs were put on an old PCI branch,
typically 100 MB/s max.  If there is a PCI RAID card in the same slot,
or, as also often happened on these older MB's, the SATA ports were
hanging off the same old/slow PCI bus, well, it could explain your results.

Which MB do you have?  Which bios rev, ...  Which raid card, how much
ram, 32 or 64 bit, yadda yadda yadda (all the details you didnt give
before).

Joe

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
       http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 866 888 3112
cell : +1 734 612 4615