[Beowulf] Followup on the ConnectX EN 10 GbE driver

[Joshua mora acosta]

Hi Joe.
Could you please get some dd to either read or write through NFS with lots of
small chunks (ie. high request rate rather than high throughput rate) in order
to find out how it correlates with the higher latency wrt Infiniband ?

Thanks,
Joshua

------ Original Message ------
Received: 11:02 PM CST, 02/13/2009
From: Joe Landman <landman at scalableinformatics.com>
To: beowulf <beowulf at beowulf.org>
Subject: [Beowulf] Followup on the ConnectX EN 10 GbE driver

> Thought a good followup was in order.
> 
> Turned out to be a firmware flash issue.  I gave access to the 
> JackRabbit to Mellanox and they flashed the firmware (wish I had known 
> this was the issue ... I must have misread the firmware download page 
> revisions).
> 
> After the update, driver loads fine.  Check your firmware.
> 
> Performance is quite good.  First time I have seen NFS (not over RDMA) 
> give 485 MB/s over plain old ethernet.
> 
> [root at jackrabbit ~]# ls -alF /data2/big.file*
> -rw-r--r-- 1 root root 34896609280 Feb 13 14:48 /data2/big.file
> -rw-r--r-- 1 root root  7918845952 Feb 13 16:31 /data2/big.file2
> 
> [root at jackrabbit ~]# mount | grep data2
> 192.168.10.157:/data on /data2 type nfs
(rw,intr,tcp,bg,addr=192.168.10.157)
> 
> [root at jackrabbit ~]# dd if=/data2/big.file of=/dev/null bs=8M
> 4160+0 records in
> 4160+0 records out
> 34896609280 bytes (35 GB) copied, 72.3984 seconds, 482 MB/s
> 
> I think we should be able to go faster.  Writes are slower (NFS over 
> RDMA also has slower writes, but not nearly as different as I saw here).
> 
> The Mellanox team was very helpful in getting this resolved.  Can't say 
> enough good things about their work to get this going.
> -- 
> Joseph Landman, Ph.D
> Founder and CEO
> Scalable Informatics LLC,
> email: landman at scalableinformatics.com
> web  : http://www.scalableinformatics.com
>         http://jackrabbit.scalableinformatics.com
> phone: +1 734 786 8423 x121
> fax  : +1 866 888 3112
> cell : +1 734 612 4615
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
>