[Beowulf] Clos networking and myrinet

[Patrick Geoffray]

Greg Lindahl wrote:
> On Wed, Jun 08, 2005 at 03:57:21PM -0400, Patrick Geoffray wrote:
> 
> 
>>I think it's typical to have several nodes talking to several *other* 
>>nodes, and that's fine with route dispersion. However, it's not typical 
>>to have several nodes talking to the *same node at the same time*.
> 
> 
> Patrick,
> 
> Obviously we aren't communicating, as I think the two are the same.
> If I am doing a halo exchange with 4 neighbors, I am talking to 4
> other nodes, and the same 4 other nodes are talking to me, too. This
> happens at roughly the same time, perhaps you are thinking of
> something different.

No, we agree. Any many-to-one communication patterns with a big "many" 
or a large message size is doomed, whatever the topology is. I just 
think they are not that frequent.

I have seen halo exchanges that where staged (you flip-flop between 2 
workspaces, and you overlap the communication of the first workspace 
with the computation on the second one), to limit the effects of 
contention. The other source of hope is that halo exchanges typically 
use small messages that do not fill the link, so if the dimension is 
small (2D), the link on the receiver will not saturate.

Patrick
-- 

Patrick Geoffray
Myricom, Inc.
http://www.myri.com