[realtek] 8139too performance - can you shed some light?

[Carlo E. Prelz]

Hello gurus.

My situation: I have to use optimally a 100MB network for transferring
multimedia material from a master machine to multiple single-board
computers.

All computers run with kernel 2.4.7.

The SBC's are powered by a NatSemi Geode CPU (Pentium-MMX class, the
core is Cyrix) @ 233MHz, have 32MB of memory and are equipped with a
RTL-8139C chip:

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

eth0: RealTek RTL8139 Fast Ethernet at 0xc2a14000, 00:d0:c9:34:97:73, IRQ 15
eth0:  Identified 8139 chip type 'RTL-8139C'
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP, IGMP
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 2048 bind 2048)
eth0: Setting 100mbps full-duplex based on auto-negotiated partner ability 41e1.

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

I equipped the master machine with two very cheap netcards, that have the
same chip (equal lines at boot time), one for connecting with the
world and the second for communications with the cluster of SBCs. 

Needing to provide the same data to all machines, I used multicast,
and UDP. I wrote a simple protocol where I can easily change the
packet size.

At the beginning of the development, before I obtained the master
machine, I developed on my home machine, talking with the SBC from a
PCI ne2000-compatible card at 10Mbps, via my cheap 4-port hub. All
went OK, with the slow rate that I expected would be multiplied by 10
with the upgrade to 100Mbps.

I set up the master machine, and its second ethernet card has been
connected to the SBC's port with a crossover cable. Normal TCP stuff
(mainly ssh) works perfectly. 

When I tested my UDP software, I discovered with dismay that most of
the packets disappeared: they were not even registered by tcpdump on
the SBC. 

I was able to reduce the packet loss to 0 by checking the output queue
on the server side with ioctl (SIOCOUTQ) and only sending when the
queue size dropped under a certain level. But then, the rate of
transmission was eventually comparable with what I obtained with
10Mbps.

While communications is just barely able to maintain its error-free
state (showing that some kind of bottleneck is reached) I can chat
with the SBC via ssh observing little or no delay.

I tried to:

- change RX_BUF_LEN_IDX to 3
- set parameter "Use PIO instead of MMIO" in kernel config

In both cases, the driver would refuse to operate.

I changed max_interrupt_work from 20 to 30 and then 60, without
appreciable changes.

I twiddled with some parameters in /proc/sys/net, also without
appreciable changes. 

I know that UDP has no assurance to deliver packets, so I expected to
lose one packet here and there, but to lose the great majority of
packets, and all this on a direct crossover cable (trying 3 different
cables, too) is a bit of an anticlimax.

My questions are:

- Did I do some stupid, evident blunder? What can my current
bottleneck be?
- What throughput can I expect from a 1-way UDP traffic between two
8139C's (and eventually, to a bunch of them on a hub/switch)? 
- Do I need to change the hardware? I know that the 8139 chip is
cheap... 

Thanks in advance. I can provide other info if requested.

Carlo Prelz

-- 
  *         Se la Strada e la sua Virtu' non fossero state messe da parte,
* K * Carlo E. Prelz - fluido@fluido.as             che bisogno ci sarebbe
  *               di parlare tanto di amore e di rettitudine? (Chuang-Tzu)