[Beowulf] fast interconnects

[Joachim Worringen]

Eugen Leitl wrote:
> Switches (crossbars) don't scale for very many ports. Especially if you have to
> do cut-through switching at those high speeds. It would be good if each NIC
> would came with an integrated switch, with enough ports to wire at least a 3d
> torus (where you route/switch messages via Bresenham).

You know that such NICS do exist for some years? If not, take a look at the SCI 
interfaces at http://www.dolphinics.com. Only a few ns per NIC routing latency. 
However, all HPC switch vendors will tell you that their switches scale great, 
and have full bisection bandwidth. And I guess they are not totally wrong.

> In regards to keeping the wires short, does this IBM trick of keeping all
> wires equal-length work well on 3d lattices, and above? This would seem to
> be a must for those coming (hopefully) Hypertransport motherboards with 
> connectors.

I assume you refer to the "interleaving" of node connections? Is this an IBM 
patent? It *might* be after all the patents I've seen so far...

The cabling is already quite tricky for 3D setups. I don't think you would like 
to go beyond. There are good reasons why nobody has done it yet.

  Joachim

-- 
Joachim Worringen - NEC C&C research lab St.Augustin
fon +49-2241-9252.20 - fax .99 - http://www.ccrl-nece.de