[Beowulf] building Infiniband 4x cluster questions - using 1 port out of 2
diep at xs4all.nl
Mon Nov 7 06:45:33 PST 2011
On Nov 7, 2011, at 12:44 PM, Robert Horton wrote:
> Most of what I know about Infiniband came from the notes at
> (or John Hearns in his previous life!).
> On Mon, 2011-11-07 at 00:01 +0100, Vincent Diepeveen wrote:
>> Do i need to connect both to the same switch?
>> So in short with infiniband you lose 2 ports of the switch to 1
>> card, is that correct?
> You probably want to just connect one port to a switch and leave the
> other one unconnected to start with.
What's the second one doing, is this just in case the switch fails, a
kind of 'backup' port?
In my naivity i had thought that both ports together formed the
bidirectional link to the switch.
So i thought that 1 port was for 10 gigabit upstream and the other
port was for 10 gigabit downstream,
did i misunderstood that?
>> Do all switches work with all cards?
>> Can you mix the many different cards that are out there of 4x
>> If not, can you mix from mellanox the different cards theirs? So mix
>> from 1 manufacturer cards?
> They will (or at least should) all work to a point but depending on
> combination you are using you may not get some features. If you
> want an
> easy life keep it all from the same manufacturer
I will load the switch (es) to the maximum number of messages a
second it can handle,
>> Does infiniband 4x work for x64 machines?
> The 4x bit is the number of links aggregated together. 4x is normal
> connections from a switch to a node, higher numbers are sometimes used
> for inter-switch links. You also need to note the data rate (eg SDR,
> DDR, QDR etc).
>> Drivers for cards now. Are those all open source, or does it require
>> payment? Is the source released of
>> all those cards drivers, and do they integrate into linux?
> You should get everything you need from the Linux kernel and / or
>> The MPI library you can use with each card is that different
>> manufacturer from manufacturer? Free to download
>> and can work with different distro's? Does it compile? Is it just a
>> library or a modified compiler?
> There are quite a lot to choose from but OpenMPI is probably a good
> starting point.
> Hope that's some help...
More information about the Beowulf