TCP is fine in Linux kernel 2.4.3

Josip Loncaric josip at
Fri Aug 10 08:32:50 PDT 2001

My TCP patches (see
which improve TCP performance in Linux kernels 2.2.x are not needed in
Linux kernel 2.4.3 (and above?).

In Linux kernels 2.0 and 2.2, the TCP has an unfavorable interaction
between sender side congestion control and receiver side delayed
acknowledgments, where delayed ACKs mimic congestion and force the
sender into repeated data stream stalls lasting 1-2 scheduler intervals
(10 or 20 milliseconds).  This creates a huge problem for two reasons:
(1) at regular intervals, small packets get delayed 1000-2000 times
longer than normal and (2) parallel applications, which use
unidirectional small packets to synchronize operation, can experience
protracted stalls lasting a minute or more because the probability of a
1000 times longer delay is proportional to the number of data paths,
which grows like n^2 where n is the number of processes involved.

For kernels 2.0 and 2.4, my patch produced 10-20 fold improvement
through a probabilistic delayed ACK policy.  By choosing immediate ACKs
with probability 1/8, the receiver could maintain the sender in a mode
where several small outgoing packets were merged into a single larger
one.  This of course increased efficiency of the transfers so that each
queued single byte TCP packet averaged 5.5 microseconds (sometimes as
low as 3.3 microseconds), whereas without packet merging 7.2
microseconds is the theoretical best time using fast ethernet.  By
contrast, the plain 2.2 kernel requires 65.2 microseconds on the average
(with great variability: standard deviation is 64.5 microseconds, and
the worst case is about 10,000 microseconds).

Alas, Linux 2.4 is less efficient in merging packets, and it typically
requires 12.3 microseconds per single byte TCP packet queued.  However,
the probabilistic delayed ACK policy does not help much, reducing this
average time to 11.6 microseconds.  Both plain 2.4 and 2.4+patch deliver
EXCELLENT consistency, with standard deviation of only about 0.1

Given all this, I do not think that patching kernel 2.4 is worth the
effort.  Some packet merging efficiency has been lost, but more
importantly, the results are now very consistent, so the likelyhood of
causing stalls in a parallel application is greatly reduced.


Dr. Josip Loncaric, Research Fellow               mailto:josip at
ICASE, Mail Stop 132C           PGP key at
NASA Langley Research Center             mailto:j.loncaric at
Hampton, VA 23681-2199, USA    Tel. +1 757 864-2192  Fax +1 757 864-6134

More information about the Beowulf mailing list