[Beowulf] RE: Network Filesystems performance (Glen Dosey)

[Glen Dosey]

David,

Thanks for the pointer. This was it. I'm now able to sequentially read
at line speed ( 1000Mb/s ) over NFS. Also, thanks for mentioning it
could be changed without a reboot. In RHEL4 I was passing it as a kernel
option and it wasn't available under /sys, making testing difficult. In
RHEL5 I can it access under /sys which made it a lot easier to test. I
wouldn't have bothered to look if you hadn't mentioned it.


On Fri, 2007-08-24 at 09:08 -0700, David Mathog wrote:
> > Glen Dosey <doseyg at r-networks.net> wrote
> > On Fri, 2007-08-24 at 08:45 -0400, Lawrence Sorrillo wrote:
> > > I am uncertain what this step does....
> > > 
> > > now we unmount the NFS share, recreate the file on the server, and
> remount
> > > it to clear the client cache but leave it cached on the server
> > > 
> > > [root at wopr1 ~]# dd if=/mnt/array3/file.dd of=/dev/null bs=4k
> > > 524287+0 records in
> > > 524287+0 records out
> > > 2147479552 bytes (2.1 GB) copied, 18.5161 seconds, 116 MB/s
> > > 
> > > 
> > > What does it accomplish?
> > 
> > I was using this step to illustrate that NFS itself is capable of
> > transferring data across the GigE network at near wires speeds. 
> > 
> > By unmounting the NFS share we clear the file from being cached in RAM
> > on the client, ensuring that it must be gotten from the server again via
> > NFS as opposed to being grabbed from RAM on the client. 
> 
> I ran into a nasty bug involving "dump" on recent kernels,
> the long and still not fully resolved story is here:
> 
> http://bugzilla.kernel.org/show_bug.cgi?id=8636
> 
> In a nutshell, dump on recent kernels runs hideously slow on (P)ATA
> disks (by a factor of 6!) because of this change in the .config file:
> 
>  -CONFIG_DEFAULT_IOSCHED="cfq"
> +CONFIG_DEFAULT_IOSCHED="anticipatory"
> 
> (cfq in the new kernels, anticipatory in the old ones). The "fix" was:
> 
> echo "anticipatory" > /sys/block/hda/queue/scheduler
> 
> Now this may be completely irrelevant, for instance, there was no
> problem with the SCSI disks I tested, but there might be with your raid
> controller.  If NFS can move the data across the network at full speed,
> but can't seem to read nearly as fast as the controller, it's possible
> that the scheduler you're using is involved.  There are 4 to choose
> from, perhaps give them all a test and see if any of them improve matters?  
> 
> Regards,
> 
> David Mathog
> mathog at caltech.edu
> Manager, Sequence Analysis Facility, Biology Division, Caltech
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf