The FNN (Flat Neighborhood Network) paradox

[Bogdan Costescu]

On Fri, 23 Feb 2001, Georgia Southern Beowulf Cluster Project wrote:

> This is the primary drawback I can think of immediately, however, as you've
> pointed out the method I described is "possible" though it may not be
> significantly useful.  I just wish to know if it is really, really possible
> and if so can it be used to study the software and situation to develop a
> much better solution (faster loading module or a daemon that can monitor for
> this situation).

Seems like you're not the only one interested in this. I had a small
private thread with somebody else, but I'll repeat my ideea 8-)
Anyway, don't expect something like: it takes xxx miliseconds.

Before using a bonded link (provided that all network drivers are already
loaded), you have to:
- insmod bonding.o (maybe from disk)
- run /sbin/ifenslave (maybe from disk) which will set the same MAC
address to all slaves; at present this operation can only be done by a
down then up sequence (Don Becker says that allowing a ioctl to change it
at will is prone to races).
- modify the routing table; this is not simple as down-ing the driver at
the previuos step will remove the route and you might get "no route to
host" type of errors before you set the new ones.

The most time consuming operation would probably be the down-up sequence.
Compared with latencies of (sub-)microseconds for hardware and even with
the higher latencies introduced by the TCP (also discussed on this list),
these operations might took ages.

The insmod/rmmod operations are also prone to races. The 2.4 line has
fixed several races in this area (just before 2.4.0), but AFAIK the 2.2
line didn't and will not.

Sincerely,

Bogdan Costescu

IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen
Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY
Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868
E-mail: Bogdan.Costescu at IWR.Uni-Heidelberg.De