[Beowulf] Clos networking and myrinet
patrick at myri.com
Wed Jun 8 14:16:08 PDT 2005
Greg Lindahl wrote:
> On Wed, Jun 08, 2005 at 03:57:21PM -0400, Patrick Geoffray wrote:
>>I think it's typical to have several nodes talking to several *other*
>>nodes, and that's fine with route dispersion. However, it's not typical
>>to have several nodes talking to the *same node at the same time*.
> Obviously we aren't communicating, as I think the two are the same.
> If I am doing a halo exchange with 4 neighbors, I am talking to 4
> other nodes, and the same 4 other nodes are talking to me, too. This
> happens at roughly the same time, perhaps you are thinking of
> something different.
No, we agree. Any many-to-one communication patterns with a big "many"
or a large message size is doomed, whatever the topology is. I just
think they are not that frequent.
I have seen halo exchanges that where staged (you flip-flop between 2
workspaces, and you overlap the communication of the first workspace
with the computation on the second one), to limit the effects of
contention. The other source of hope is that halo exchanges typically
use small messages that do not fill the link, so if the dimension is
small (2D), the link on the receiver will not saturate.
More information about the Beowulf