[Beowulf] building Infiniband 4x cluster questions

[Gilad Shainer]

> > Do i need to connect both to the same switch?
> > So in short with infiniband you lose 2 ports  of the switch to 1 card,
> > is that correct?
> 
> You probably want to just connect one port to a switch and leave the other
> one unconnected to start with.

Correct. You only need to connect one port. The second port can be used for performance increase or fail over for example.
 
> > CARDS:
> > Do all switches work with all cards?
> > Can you mix the many different cards that are out there of 4x
> > infiniband?
> > If not, can you mix from mellanox the different cards theirs? So mix
> > from 1 manufacturer cards?
> 
> They will (or at least should) all work to a point but depending on what
> combination you are using you may not get some features. If you want an
> easy life keep it all from the same manufacturer
 
All cards and switches build according to the spec will work. 

> > Does infiniband 4x work for x64 machines?
> 
> The 4x bit is the number of links aggregated together. 4x is normal for
> connections from a switch to a node, higher numbers are sometimes used
> for inter-switch links. You also need to note the data rate (eg SDR, DDR, QDR
> etc).

4X means 4 network lanes (same as the PCIe convention - PCIe x4, x8 etc.). It is related to the port speed, not the server architecture. Most of the InfiniBand port out there are 4X

> 
> > DRIVERS:
> > Drivers for cards now. Are those all open source, or does it require
> > payment? Is the source released of all those cards drivers, and do
> > they integrate into linux?
> 
> You should get everything you need from the Linux kernel and / or OFED.
 
You can also find the drivers on the vendors sites. Not sure about the rest, but for the Mellanox case it is open source and free - both for Linux and Windows


Gilad