[Beowulf] building Infiniband 4x cluster questions - using 1 port out of 2
Vincent Diepeveen
diep at xs4all.nl
Mon Nov 7 06:45:33 PST 2011
On Nov 7, 2011, at 12:44 PM, Robert Horton wrote:
> Hi,
>
> Most of what I know about Infiniband came from the notes at
> http://www.hpcadvisorycouncil.com/events/switzerland_workshop/
> agenda.php
> (or John Hearns in his previous life!).
>
> On Mon, 2011-11-07 at 00:01 +0100, Vincent Diepeveen wrote:
>> Do i need to connect both to the same switch?
>> So in short with infiniband you lose 2 ports of the switch to 1
>> card, is that correct?
>
> You probably want to just connect one port to a switch and leave the
> other one unconnected to start with.
What's the second one doing, is this just in case the switch fails, a
kind of 'backup' port?
In my naivity i had thought that both ports together formed the
bidirectional link to the switch.
So i thought that 1 port was for 10 gigabit upstream and the other
port was for 10 gigabit downstream,
did i misunderstood that?
>
>> CARDS:
>> Do all switches work with all cards?
>> Can you mix the many different cards that are out there of 4x
>> infiniband?
>> If not, can you mix from mellanox the different cards theirs? So mix
>> from 1 manufacturer cards?
>
> They will (or at least should) all work to a point but depending on
> what
> combination you are using you may not get some features. If you
> want an
> easy life keep it all from the same manufacturer
I will load the switch (es) to the maximum number of messages a
second it can handle,
>
>> Does infiniband 4x work for x64 machines?
>
> The 4x bit is the number of links aggregated together. 4x is normal
> for
> connections from a switch to a node, higher numbers are sometimes used
> for inter-switch links. You also need to note the data rate (eg SDR,
> DDR, QDR etc).
>
>> DRIVERS:
>> Drivers for cards now. Are those all open source, or does it require
>> payment? Is the source released of
>> all those cards drivers, and do they integrate into linux?
>
> You should get everything you need from the Linux kernel and / or
> OFED.
>
>> MPI:
>> The MPI library you can use with each card is that different
>> manufacturer from manufacturer? Free to download
>> and can work with different distro's? Does it compile? Is it just a
>> library or a modified compiler?
>
> There are quite a lot to choose from but OpenMPI is probably a good
> starting point.
>
> Hope that's some help...
>
> Rob
>
More information about the Beowulf
mailing list