High Speed Beowulf Cluster using gigabit ethernet

[Felix Rauch]

On Wed, 20 Feb 2002, Velocet wrote:
> Anyone else got NS83820s with some stats? 64 bit vs 32bit? 66Mhz vs 32Mhz?

[Probably I missed part of the discuss since I just came back from
vacation.]

We do have some cards with DP82820 chips on them (the box says "Asante
FriendlyNET GigaNIX Ethernet Adapter"). The performance we got with
the standard driver as well as with a driver developed by a student
was disappointing. We got much better results with older Hamachi
GNIC-II Gigabit Ethernet cards. Unfortunately I don't have the results
at hand right now.

However, what a student at our laboratory found was that the chip has
several hardware errors which are annoying when trying to use advanced
features. For those interested in the details I will copy the detailed
description of the errors below.

- Felix

--- quote from another mail ---

The hardware bugs of the DP83820 are probably not so relevant for a
standard linux driver, but I will give you a general description
anyway. They were dicovered by Christian Widmer.

- VLAN-tag stripping: The hardware can automatically add and remove
  VLAN-tags. It appears that there is a problem with short frames when
  VLAN-tags are automatically removed from a frame on receiption. The
  chip does the "short frame check" after removing the VLAN-tag
  without remembering that it already removed something from the
  frame. The frame is now shorter than the minimal Ethernet frame
  length (if it was exactly the minimal length before stripping the
  VLAN-tag). If it's set up to discard too short frames, it simply
  drops the frame, even though it was actually _not_ too short before
  removing the VLAN-tag. So, on the receiver side, there will never be
  a receive interrupt because even the ARP packets are all dropped.

- Multi descriptor frames: Theoreticaly the chip can have multiple
  descriptors for one frame. This would make it possible to
  automatically scatter/gather ethernet-, IP- and other headers from
  the payload. This seems to work on the sender side, but only with
  some restrictions on the receiver side: It appears that the chip can
  only switch to the next descriptor on 16-bytes boundaries. Tests
  showed that if the first descriptor has 14 bytes room, there are
  nevertheless 16 bytes copied to the descriptor. This corrupts data
  after the descriptor and separates the received data at unwanted
  boundaries.

- Priority queues: The DP83820 has up to four RX/TX rings and it's
  possible to choose how many of them should be used. Based on the
  priority of the VID in the VLAN-tag, the chip then stores received
  frames in the different queues. The documentation tells us that
  frames without VLAN-tag are always stored in queue 0, which is not
  true. See the following tables:

  Using 2 priority queues (correct behaviour):
  Queue         No VLAN-tag     VID priority
                                0 1 2 3 4 5 6 7
  0             x               x x x x
  1                                     x x x x

  Using 3 priority queues (correct behaviour):
  Queue         No VLAN-tag     VID priority
                                0 1 2 3 4 5 6 7
  0             x               x x x x
  1                                     x x
  2                                         x x

  Using 4 priority queues (*in*correct behaviour):
  Queue         No VLAN-tag     VID priority
                                0 1 2 3 4 5 6 7
  0                               x x
  1             x               x     x
  2                                     x x
  3                                         x x

  This looks like a bit error to me.

I hope this information is helpful to you. If you know anything more
about special behaviours of this chip, we would appreciate a hint.

--- end quote ---

-- 
Felix Rauch                      | Email: rauch at inf.ethz.ch
Institute for Computer Systems   | Homepage: http://www.cs.inf.ethz.ch/~rauch/
ETH Zentrum / RZ H18             | Phone: ++41 1 632 7489
CH - 8092 Zuerich / Switzerland  | Fax:   ++41 1 632 1307