[Beowulf] 10G networking?

[Gavin W. Burris]

Hi, Mark.

Here are some tests,  Let me know if you'd like something specific.

# lspci |grep Ether
    01:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)
    01:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)
    03:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)
    03:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)
# ping -M do -s 8972 hpcc001-i -c 60
    60 packets transmitted, 60 received, 0% packet loss, time 58999ms
    rtt min/avg/max/mdev = 0.116/0.137/0.191/0.024 ms
# ping hpcc001-i -c 60
    60 packets transmitted, 60 received, 0% packet loss, time 58999ms
    rtt min/avg/max/mdev = 0.084/0.104/0.176/0.022 ms
# iperf3 -c hpcc001-i -p1234 -t 60 -i 10
    [ ID] Interval           Transfer     Bandwidth       Retr
    [  5]   0.00-60.04  sec  69.0 GBytes  9.87 Gbits/sec   72             sender
    [  5]   0.00-60.04  sec  69.0 GBytes  9.87 Gbits/sec                  receiver


On 10:09AM Fri 01/23/15 -0500, Mark Hahn wrote:
> >We have been using 10g CX4 for the past 7yrs for our cluster core
> >network and are just going fully into 10gT. But our focus is more on
> >bandwidth/HTC instead of HPC, i.e. we do not care so much about latencies.
> 
> I'm sorry if I was unclear: I'm interested specifically in whether recent
> developments change the basic 10G picture.  we have cx4 as well,
> and lots of IB, but all our 10g is oldish stuff.  I guess I'm kind of hoping
> to hear that switch, nic, driver improvements have changed the picture from
> years ago, when the best you could do with 10g was ~7 us
> latency.
> 
> >Do you have any particular tests in mind, we can certainly look into it
> >and report back on our findings.
> 
> min packet ping-pong latency is the standard metric, though packets/sec
> rate might be interesting as well.
> 
> thanks, mark hahn.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Gavin W. Burris
Senior Project Leader for Research Computing
The Wharton School
University of Pennsylvania