[Beowulf] mlx 10g ethernet

[Alex Chekholko]

The ESNet pages are pretty good:
https://fasterdata.es.net/host-tuning/
https://fasterdata.es.net/network-tuning/

Since you're on a local low-latency link, all the stuff about
high-bandwidth high-latency links doesn't apply to you, but the host tuning
parameters at least give you somewhere to look.

I recommend iperf for testing before IMB, make sure you can get line rate
throughput with an iperf test.

On Fri, Aug 4, 2017 at 8:52 AM Michael Di Domenico <mdidomenico4 at gmail.com>
wrote:

> does anyone have any additional tuning tips beyond the random things
> on the net for 10g ethernet tuning?  i have two identical servers
> (Quanta Q71L-4U) attached to a single cisco 10g switch, using
> identical (Mellanox MNPH28B-XTC) cards.  when i run mpi bandwidth
> tests between the two cards, i top out at 200-300MB/sec using IMB
> sendrecv.  if i use lustre lnet selftests, i can crank the max to
> 500MB/sec, but i can't seem to crest this.
>
> i've monkied around with the sysctl params and the cards are in x8
> slots at 2.5GT/s.  i think they should be capable of driving at
> ~900MB/sec, but i can't seem to get there.  i know i'm missing
> something...
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20170804/df01a326/attachment.html>