LAM: Multiple NICs in SMP nodes

[Douglas Eadline]

On Wed, 16 Aug 2000, David van der Spoel wrote:

> Hi,
> 
> This may be a bit off-topic (OS / MPI) but anyway...
> I have a couple (4, soon 8) of SMP machines (dual P3) running RH linux. To
> improve network performance I bought extra network cards, so that all
> machines have two. My main application (molecular simulation) communicates
> in a ring topology most of the time. Dual network cards could be connected
> in a point to point topology for optimal communication, but other
> (non-ring) communication is sometimes necessary as well, so everything is
> connected through a switch running at 100 Mbit full duplex. 
> 
> Now here are the questions:
> Is there a way of binding processes to network cards (in general not in
> Linux as far as I know...), or alternatively,  is there a way to have a
> persistant TCP connection within LAM? As a third alternative: is there
> anyone that has experimented with channel bonding in recent (2.2.16+)
> Linux kernels? Does one need a special switch to use channel bonding?


The following is on PII-Linux RH6.2 system.

I looked at this a while ago and found I could not get LAM to do it.
My idea was that since there are two CPUs on the motherboard and
two network cards (each running a separate network), I tried to
see if I could boot LAM to use both networks. For instance,

I had 4 systems called coyote1, coyote2, coyote3, coyote4
On the second network, there were called coyote1a, coyote2a, 
coyote3a, coyote4a.

I tried a boot schema:

coyote1
coyote2
coyote3
coyote4
coyote1a
coyote2a
coyote3a
coyote4a

So when LAM tried to start on coyote1a it complained about
the daemon already running. My goal was to fool LAM
and make 4 dual machines look like 8 singles. 

The best I came up with was to do this:

coyote1
coyote2a
coyote3
coyote4a

This way if coyote1 needed to send to all the nodes, it could
use two interfaces at the same time.  I tried running the NASA
parallel benchmarks with this, but found no big performance gain.

I have tried channel bonding, it works. However, if you need 
better latency, channel bonding will provide the
same a single network. Throughput however, can be almost double 
a single network card.

Just some random thoughts

Doug


> 
> 
> Groeten, David.
> ________________________________________________________________________
> Dr. David van der Spoel		Biomedical center, Dept. of Biochemistry
> s-mail:	Husargatan 3, Box 576,  75123 Uppsala, Sweden
> e-mail: spoel at xray.bmc.uu.se	www: http://zorn.bmc.uu.se/~spoel
> phone:	46 18 471 4205		fax: 46 18 511 755
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 
> _______________________________________________
> This list is archived at http://www.mpi.nd.edu/MailArchives/lam/
> 

-- 
-------------------------------------------------------------------
Paralogic, Inc.           |     PEAK     |      Voice:+610.814.2800
130 Webster Street        |   PARALLEL   |        Fax:+610.814.5844
Bethlehem, PA 18015 USA   |  PERFORMANCE |    http://www.plogic.com
-------------------------------------------------------------------