Channel-bonding progress
Mike Weller
weller at zyvex.com
Wed Aug 8 23:05:15 PDT 2001
I wrote:
>Hi,
>
>I think I managed to solve the problem. I noticed that in
>my "ifconfig" output, there's a lot of "overruns" packets reported.
>
>Also, someone mentioned the possibility of it not arp'ing through
>bond0.
>
>I did 2 things. I'm not sure what works yet, but here are my
>preliminary results. I can achieve 180-200Mbps in bonding mode if I
>change the MTU packet size from 1500 to 1200. I also removed the eth1
>and eth0 entries in the routing table to ensure that it uses bond0.
>I'm going to install a better bandwidth benchmark and possibly search
>for the optimal MTU size and post the results. Here's the script that
>I used to get the 180-200Mbps with FTP:
Using a sniffer, I've determined that the actual packet sizes of eth0
and eth1 used are based on the MTU of bond0. For instance, if I set
the MTU of eth0 and eth1 to 100, and bond0 to 1000, the packet sizes
sniffed on eth0 are close to 1000. If I drop it to 500, the packets
on eth0 and eth1 drop to 500.
I also noticed that whenever I made a change to the routing table or
the MTU size of the interface, there is temporary instability, but
for the trials after that, I get consistency (99% confidence).
Can anyone explain the temporary instability?
I conclude that the MTU sizes of eth0 and eth1 are irrelevant, and
that having eth0 and eth1 in the routing table is irrelevant.
Using 2 crossover cables (no switch), along with the command:
"netperf -H 10.0.0.2 -i 10,1", here's a table of results.
1100 seems to be the optimal MTU size for bond0. As the packet size
increases beyond 1100, a large number of packets are "overrun",
resulting in poor bandwidth. I assume that the bandwidth drops
off below 1100 because more packets are required to be sent (thus,
more overhead).
MTU-bond0 MTU-eth0 MTU-eth1 Transfer Rate(Mbps) Confidence(%) Overruns (%)
-------- -------- -------- ------------------ ------------- ----------
(keeping eth+eth0 in routing tables:)
1000 1000 1000 159.53 99 0
1000 1000 1000 159.66 99 0
(removing eth1+eth0 from routing table:)
1000 1000 1000 57.79 low ?
1000 1000 1000 163.27 99 0
(conclusion: having eth0+eth1 in routing table doesnt hurt)
1000 100 100 109.80 low ?
1000 100 100 163.74 99 0
1000 100 100 164.98 99 0
1000 500 500 67.93 ? 0
1000 500 500 163.61 99 0
(conclusion: the MTU sizes of eth0 and eth1 don't matter)
...
1500 750 750 15.84 40 2
1500 750 750 37.14 58 5
1400 700 700 20.09 22 4
1400 700 700 20.77 72 6
1300 650 650 27.73 25 4
1300 650 650 24.28 46 5
1200 600 600 39.44 39 3
1200 600 600 51.19 12 2
1100 550 550 67.10 148 .07
1100 550 550 168.75 99 .08
1100 550 550 168.02 99 .08
1000 500 500 43.93 29 .001
1000 500 500 163.63 99 .001
900 450 450 37.85 64 0!
900 450 450 156.82 99 0
800 400 400 25.91 15 0
800 400 400 126.66 61 0
800 400 400 140.08 8 0
800 400 400 142.27 99 0
700 350 350 101.49 96 0
600 300 300 109.28 99 0
400 200 200 76.37 99 0
1100 550 550 169.03 99 .08
(disconnected crossovers, connected to switch instead)
1100 550 550 165.42 8 .05
1100 550 550 166.44 8 .06
--
Michael J. Weller, M.Sc. office: (972) 235-7881 x.242
weller at zyvex.com cell: (214) 616-6340
Zyvex Corp., 1321 N Plano facsimile: (972) 235-7882
Richardson, TX 75081 icq: 6180540
More information about the Beowulf
mailing list