[tulip] Re: [eepro100] Re: True on TRANSMIT ERROR TIMEOUT

[Andrew Morton]

[ I really wish I hadn't done that reply-to-all on
  Tuesday.  Can we please henceforth keep this on
  netdev? ]


Bogdan Costescu wrote:
> 
> On Tue, 13 Jun 2000, Andrew Morton wrote:
> 
> > And there are many, many times when a wedged driver can be resurrected
> > by a down/up or a rmmod/insmod.  This means that the driver _could_ have
> > automtically recovered in tx_timeout, but it simply did not do so.
> 
> I beg to disagree. The above mentioned operations are doing much more than
> handling TX timeouts: register/unregister IRQ, get/release memory, set up
> (from scratch) the Tx and Rx buffers, media selection... The tx_timeout
> routine should only recover from a Tx timeout! What you propose is
> something like calling xxx_open() from tx_timeout... if I understand it
> right.

Well, I didn't say "let's put in lots of bugs" :)

My point is very simple:

- Drivers and/or NICs are hanging
- The hangs are fixed by down+up or rmmod+insmod

Hence, the hangs _could_ be unhung by appropriate action
in the tx timeout!

This would be a great step forward.  A sub-second hiccup and
a few dropped packets versus a complete system outage.

It's not pretty, it shouldn't be necessary, but geeze, it's
better than what we have now.  Without access to the ASIC
designers and everything else, we may never resolve these
problems.  And can we be sure that the closed-source
vendor-developed drivers don't _currently_ reset the
crap out of the NIC when it goes unresponsive?

> > Can anyone suggest a reason why we _shouldn't_ simply reset the NIC to
> > the utmost possible extent in tx_timeout?  Restart media selection,
> > reinitialise ring buffers, etc, etc?
> 
> Simply because you need to know the exact state of the card in order to
> save the relevant parts of it, reset and then load the card with
> the previous values (of course, you don't need to re-load the parts
> which created the problem).

mm.  This is a matter of keeping appropriate state in the
dev private space.  True, the effects of 'mii-diag -F -A'
will be lost, but they'll also be lost across an up+down
or a reboot, no?

> Think of the 3c59x driver: you might want to
> save the Tx/Rx threshold related values, poll interval values and so on.
> This is only efficient if you keep most of them to the default (as
> power-on) values.
> For media selection, xxx_timer should take care of this, there should be
> no need for tx_timeout to handle media changes. When the transmission is
> stopped because of a media change, the xxx_timer routine should take care
> of the media state, then tx_timeout routine should reset the transmitter
> and everything should be working again. Also adding media related logic to
> tx_timeout raises the problem of protecting the access to the media
> related registers WRT xxx_timer...

These failures appear to correlate with high traffic which,
I suggest, points the finger at driver/hardware bugs rather
than media selection probs.

> I don't think that flushing the queue (as somebody else from this thread
> suggested) is a good ideea as you loose a lot of packets (usually 16).

A typical tx_timeout routine will at present cause up to
16 packets to be retransmitted (duplicated).  Packet loss
is not a problem (particularly when compared with loss
of an interface).

> And
> usually the buffers themselves have no relation with the
> transmission/reception logic - you might need to restart the transmitter
> and/or the DMA engine and maybe write again the head buffer to it -
> that's all.
> 
> Now, coming back to the initial message about "correlated" reports of Tx
> timeouts using some boards/drivers, I don't think this is true. Most of
> the problems are in fact related to media selection (when autonegotiation
> fails and the boards cannot transmit properly) and real timeouts (e.g.
> caused by collisions) that are board specific. In this case, there is no
> general rule that has to be applied - the hardware has to be driven "The
> Right Way" (which might not be even documented).

Perhaps an appropriate detection algorithm is to do a
radical reset & reinitialisation if two tx timeout occur
with no intervening tx interrupts.

Have you played with the tx timeout interval?  Even with
14 packet tx interrupt mitigation, it's very, very hard
to force a tx timeout on a 10baseT LAN with the timeout
set to just 20 milliseconds.  We're using 400!


I dunno.  I just get the impression from the mailing
lists that the network drivers are a weak spot for
Linux, and we're letting the whole game down.  Help
me out here...

Let me quote "Brian" from last week:

  ``The web servers (~10Mb/s) last about 3 days on
    <NIC>.  With all the Linux zealots talking about
    taking on Sun and the "enterprise," I'd drop
    dead laughing if they weren't my servers.''

Ouch.