[Beowulf] how fast can NFS run?

[Craig Tierney]

Peter Kjellstrom wrote:

>On Wednesday 01 February 2006 05:13, Bruce Allen wrote:
>  
>
>>I'd like to know the fastest that anyone has seen an NFS server run, over
>>either a 10Gb/s ethernet link or a handful of link aggregated
>>(channel-bonded) Gb/s ethernet lines.
>>
>>This would be with a small number of clients making large file sequential
>>reads from the same NFS host/server.  Please assume that the NFS server
>>has 'infinitely fast' disks.
>>    
>>
>
>First, don't hijack threads, can't you guys have any mercy on us with thread 
>capable e-mail clients? ;-)
>
>I do think that you could get NFS up over the 100 MiB/sec mark, but I also 
>think it would be alot easier with lustre (www.lustre.org) since then you 
>could use a few GigE connected servers and not depend on extreme speed and 
>tuning on one single server...
>  
>
Lustre would not be easier.  NFS is supported on virtually all operating 
systems. 
Also, it has been around a lot longer and is more stable.  And if you 
want to do
something more than read and write large sequential files, you better 
stick with NFS (or look at
other less mature distributed filesystems).  Lustre (and most HPC 
filesystems)
has very poor meta-data performance.  You wouldn't want to compile codes,
work with NetCDF, or other small file operations on a Lustre 
filesystem.  NFS is better,
it just may not scale if you want to get to the multi-GB/s range.  Other 
commercial
products can get you there (Polyserve and Ibrix are two examples).

>If you go down the nfs path you might want to google on, among other things, 
>NFS over rdma, NFS infiniband, ...
>  
>
This could be good, but the established technology is going to easily
get performance over 100 MB/s without breaking the bank.
Only problem is that you still have to support it yourself, and in the
long run is this going to be a cheaper solution?

Craig



>just my 0010 cents,
> Peter
>
>  
>
>>I am told by one vendor that "NFS can't run faster than 100MB/sec".  I
>>don't understand or believe this.  If the server's local disks can
>>read/write at 300MB/s and the networking can run substantially faster than
>>100 MB/s, I don't see any constraint to faster operation.  But perhaps
>>someone on this list can provide real-world data (or say why it can't
>>work).
>>
>>Note: I am free to use modern versions of the NFS protocol, jumbo frames,
>>large rsize/wsize, etc.
>>
>>Cheers,
>> 	Bruce
>>    
>>
>
>  
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org
>To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>  
>