[Beowulf] 10G networking?

[Joe Landman]

On 01/23/2015 12:14 PM, Gavin W. Burris wrote:
>
> On 12:04PM Fri 01/23/15 -0500, Joe Landman wrote:
>> Don't use ping for HPC latency/throughput tests.
>
> What is your preferred benchmark, Joe?
>

Interesting.

Ping isn't a benchmark.  Its a very basic test of connectivity, packet 
transfer and routing.  HPC folks generally use it for baseline sanity 
checks, not performance.

HPC folks would likely use an MPI benchmark similar to 
http://mvapich.cse.ohio-state.edu/benchmarks/ for application stack 
performance similar to what Mark was talking about ("min packet 
ping-pong latency is the standard metric, though packets/sec
rate might be interesting as well").

Mark is dead on correct pointing out that ping-pong latency is one of 
the standard metrics.  It is "roughly" correlated with latency sensitive 
code performance, at least the sections that are latency sensitive.

You could use netperf or iperf for throughput tests on ethernet.  I use 
the latter to see the effects of scaling.  I usually run it as

	iperf -t 60 -P 4 ...

You can also use other similar tests, and there are a fairly wide 
ranging number of them.

For IB and RoCE/iwarp you can use the rdma diagnostics, which depending 
upon your distribution, could be named things like (regex)

(ib_){0,1}[rdma|read|send|write]_[lat|bw|(x){0,1}server|(x){0,1}client]

[debian based]
root at usn-ramboot:~# ib_
ib_clock_test         ib_rdma_lat           ib_read_lat 
ib_send_lat           ib_write_bw_postlist
ib_rdma_bw            ib_read_bw            ib_send_bw 
ib_write_bw           ib_write_lat

[CentOS based]
[root at CentOS6-build ~]# rdma_
rdma_client   rdma_server   rdma_xclient  rdma_xserver

The man pages and help are useful for these.

But, not Gavin, ping isn't normally reported when one talks about 
ping-pong.  See the OSU link for the actual test.  Specifically, 
osu_latency and osu_latency_mt for contention.


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
e: landman at scalableinformatics.com
w: http://scalableinformatics.com
t: @scalableinfo
p: +1 734 786 8423 x121
c: +1 734 612 4615