[Beowulf] fast interconnects

[Patrick Geoffray]

Hi Mark,

Mark Hahn wrote:
> 	- isn't Mellanox still the sole source for IB chips, for both
> 	nics and switches?  this seems odd if it's a thriving ecosystem.
> 	no offense intended!  yes, I know quadrics/SGI/SCI/Myri are all
> 	also sole-source.  but compared to the eth world...

As far as I know, Mellanox is the sole source for the switch chips. For 
the NICs:
* Mellanox chips is used in the vast majority of IB NICs, either the 
ones sold by resellers or directly by Mellanox.
* IBM has a chip but it's only for GX bus (internal Power5 bus). They 
may do a PCIE chip eventually, I don't know.
* QLogic/Pathscale chip implements a subset of IB in hardware, the rest 
of the spec is emulated in software (corrections welcome), but it's 
technically an IB chip.

So, there is a beginning of an ecosystem for the NICs, but not the switches.

In the Ethernet world, this is crazy. Even for the low-latency Ethernet 
crossbars, there are already chips from Fujitsu and Fulcrum (and at 
least one more source). Volume will depend on the price of course, but 
it's already quite aggressive in my opinion (Fulcrum was quoting $500 
for 24-port 10GigE crossbar around SC05).

> 	- IB seems to be squeezed between cheap 1G (which is dirt cheap
> 	and The Right Choice for most clusters), and the onset of 10G 
> 	(which could satisfy the subset of bandwidth-intense clusters.)  
> 	would there be a reason to choose IB if the eth world suddenly 
> 	started	shipping 10gbase-t in volume with switches costing, say,
> 	$1000/pt?

Hard to say a time frame, it all depends on the ramping up of low 
latency 10GigE switches. If the pricing is what is announced, it will 
definitively fit in the $1000 per port envelop.

Patrick