[Beowulf] Questions about upgrading InfiniBand

[Mark Hahn]

> Aggregation spine? Can you tell me more about that? Can you give me a
> part/model number?

spine is just the term for the trunk of a fat tree.  usually the per-rack 
switches are called leaves since if nothing else, they may not be at 
the top of the rack, or there may be more than one per rack...

the good thing about the leaf/spine approach is that it's modular,
and possibly less vendor-locked-in. 
(not that IB is really multi-vendor anyway).

cable-wise, I'm not sure leaf-spine really wins, since you can think of 
a chassis switch as a leaf-spine with FR4 rather than CX4.  AFAIKT, the 
same radix-36 switch is used in each.

spine/leave can be distributed so that there's no one place where you 
get too many cables.

"less than fully fat" fabrics seem to be pretty common when taking 
the modular approach.  for instance, if a 36x switch is split into 
24 down (node) and 12 up, you can put two back-to-back, or have 
three going into a single spine switch, or more going into multiple
spines.  you could even have some racks with more spineward links.
and in your case, you could vary the number of links going to your 
existing chassis switch (though it probably shouldn't be the spine
since all its links are slower...)

-mark