No subject

computation time to communication time for even the smallest mesh of
around 100 to 500:1, more for the larger meshes.

If your application is CPU bound with a comp:comm time ratio like 1000:1
with a fairly small serial fraction, then it will get pretty much linear
speedup out to eight nodes.  If you increase it to 2000:1, then you'll
get pretty much linear speedup out to eight nodes.  If you increase it
to 10^6:1 you'll get pretty much linear speedup out to eight nodes.  All
you're observing is that communications are irrelevant to your
(essentially coarse grained) parallel application, at least for the
switched 100BT that forms your basic network and granularities of your
code.

In which case yes, channel bonding is way overkill for your application,
at least at the ranges you portray.  Instead of buying multiple switches
and trying to build fatter pipes, buy bigger switches and more nodes.
Only when/where you start to see some sort of saturation and bending
over of the scaling curve (like that which might be visible for 9 nodes
in the 480x50 mesh) will fatter data pipes be useful to you.

At that point, what increasing the network bandwidth WILL do for you is
increase the number of nodes you can apply to the problem and still get
(nearly) linear speedup.  So on the 480x50 mesh, channel bonding and
higher bandwidth keeps you near-linear a bit further out (from the look
of this curve the serial fraction of your code is starting to compare to
the parallel fraction, causing saturation -- maybe -- in the 8-9 node
range, if this isn't an artifact of poor statistics).  Even this isn't
strongly affected by mere doubling (and doesn't much affect the slope of
this curve < 1, which is what I'm using to guess that the serial
fraction is becoming important) -- it is more of a log scale kind of
thing.  That is, increasing network speed by a factor of 10 might buy
you a factor of two or three extension of the linear scaling region.

To understand all this read the section on Amdahl's law, etc. (and look
at the equations and figures) in the online beowulf book on brahma:

  http://www.phy.duke.edu/brahma

  b) The other possibility is that your communication pattern is
dominated by tiny messages (still) interspersed with lots of
computation.  In this case your communications is probably latency
dominated, not bandwidth dominated.  Fattening the pipe won't help this;
adding an extra channel will if anything INCREASE the latency (and hence
slow the time per message).  I only mention this because it is possible
that you have lots of small messages whose aggregate time is still in
the 1000:1 range for your larger meshes.  In this case channel bonding
would be a bad idea in general, not just not worth it for the scaling
ranges you've achieved so far.  However, if you rearrange your
application to use bigger messages, it might once again be worthwhile.

Hope this helps.  Your scaling is really rather good out to 8-10 nodes
-- eyeballing it it looks impossibly good for the largest mesh, but I
rather hope this is because you are plotting the results for single run
times and better statistics would make the line behave.  Although there
>>are<< nonlinear breakpoints -- breaking the bigger mesh up means more
of it runs in cache, which can actually yield better CPU rates (which
then decrease the comp:comm ratio and really make a priori estimation of
the speedup curve a bitch, actually:-).

    rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu