[Beowulf] Mutiple IB networks in one cluster

[Prentice Bisbal]

Beowulfers,

I was talking to a colleague the other day about cluster architecture 
and big data, and this colleague was thinking that it would be good to 
have two separate FDR IB clusters within a single cluster: one for 
message-passing, and the other purely for data movement. I'm a bit 
skeptical of this myself. I was always under the impression that IB has 
more than enough bandwidth for message-passing and I/O. I have some 
questions about this idea:

1. Does this make sense?

2. Does anyone have first hand experience with doing this, or can point 
me to someone who does (articles on line, papers on the topic will 
suffice)?

3. Would the present any issues for managing the fabric? I know IB is 
designed to detect loops automatically, but what about making sure  
certain traffic stays on certain IB interfaces.

4. Since IB uses cross-bar switches (please correct me if I'm wrong), we 
wouldn't need to duplicate switchgear, just double IB connections on 
each host, correct?

-- 
Prentice