[Beowulf] building Infiniband 4x cluster questions
robh at dongle.org.uk
Mon Nov 7 03:44:49 PST 2011
Most of what I know about Infiniband came from the notes at
(or John Hearns in his previous life!).
On Mon, 2011-11-07 at 00:01 +0100, Vincent Diepeveen wrote:
> Do i need to connect both to the same switch?
> So in short with infiniband you lose 2 ports of the switch to 1
> card, is that correct?
You probably want to just connect one port to a switch and leave the
other one unconnected to start with.
> Do all switches work with all cards?
> Can you mix the many different cards that are out there of 4x
> If not, can you mix from mellanox the different cards theirs? So mix
> from 1 manufacturer cards?
They will (or at least should) all work to a point but depending on what
combination you are using you may not get some features. If you want an
easy life keep it all from the same manufacturer
> Does infiniband 4x work for x64 machines?
The 4x bit is the number of links aggregated together. 4x is normal for
connections from a switch to a node, higher numbers are sometimes used
for inter-switch links. You also need to note the data rate (eg SDR,
DDR, QDR etc).
> Drivers for cards now. Are those all open source, or does it require
> payment? Is the source released of
> all those cards drivers, and do they integrate into linux?
You should get everything you need from the Linux kernel and / or OFED.
> The MPI library you can use with each card is that different
> manufacturer from manufacturer? Free to download
> and can work with different distro's? Does it compile? Is it just a
> library or a modified compiler?
There are quite a lot to choose from but OpenMPI is probably a good
Hope that's some help...
More information about the Beowulf