[Beowulf] Help for terrible NFS write performance

[Joe Landman]

Orion Poplawski wrote:
> On 08/21/2009 11:38 AM, Joe Landman wrote:
>> Orion Poplawski wrote:
>>> /etc/exports:
>>> /export *.cora.nwra.com(rw,sync,fsid=0)
>>> /export/cora6 *.cora.nwra.com(rw,sync,nohide)
>>> /export/working *.cora.nwra.com(rw,sync,nohide)
>>
>> Ok. There it is... Sync.
>>
>> Don't need to see anything else.
>>
>> That is it.
> 
> Okay, how afraid should I be of using async?  man exports states:
> 
>        async  This option allows the NFS server to violate the NFS protocol
>               and  reply to requests before any changes made by that 
> request
>               have been committed to stable storage (e.g. disc drive).
> 
>               Using this option might improve  performance  with version  2
>               only,  but  at the cost that an unclean server restart 
> (i.e. a
>               crash) can cause data to be lost or corrupted.

If your NFS server crashes, you *could* lose data.  Not you *will* lose 
data.  But since you are using md raid without battery backed cache, 
chances of data loss could be higher (all the RAID calculations happen 
in RAM).  The file system could be properly  resilient.  What would you 
do if the server crashed?  Would you have users restart their runs?

Its all a question of risk, real and imagined (or more correctly ... 
over-emphasized).

> I tend to like to avoid data loss or corruption.

So do most people.  If your node crashes, do you get data loss?  If your 
server crashes, will you get data loss?  I am guessing that if you do an

	hdparm -W /dev/sd*

you will find your write caching on, on each drive.  If so, sync is less 
of an issue, and you have bigger worries.

This is BTW an area where having a big fast RAID card with a large cache 
is a definite advantage over md raid.  Data in a battery backed RAID 
cache won't go away under a reboot/crash.

> 
> Thanks!
> 


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/jackrabbit
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615