Ethernet-Gigabit-Myrinet in one system

[Donald Becker]

On Fri, 19 Jul 2002, Ivan Oleynik wrote:

> Tyan i7500 motherboards contain built-in Gigabit Ethernet and Fast
> Ethernet. What are the best ways to utilize these additional resources in
> the system that will contain Myrinet interconnect as a base communication
> infrastructure. Are there any substantial advantages to link the nodes via
> additional Etherner or Gigabit switch to get substantial speed up of
> communication traffic?

Run large block traffic (usually file systems) over gigabit Ethernet,
and small or latency sensitive message traffic over Myrinet.  Gigabit
Ethernet has dropped in price and machines are fast enough that using
both is now likely a performance/cost effective approach.  An advantage
for a prototype system is that you can do the quick tradeoff test by
switching to use just one or the other.

> Does this require additional software/hardware? Are
> there any problems with this solution?

Hardware: obviously, you need a gigabit Ethernet switch for gigabit
speeds.  The cost for 24 ports and less is running around $100/port.
More than 24 ports on a switch and the cost increases significantly.  Up
to about 40 ports you can gang two switches, past that and bi-section
bandwidth starts to impact performance of linking multiple independent
switches.

Software: as usual, it depends on your system and application.  We would
charge extra for engineering hours to customize a system like this
because there are so many application variables.  With a Scyld Beowulf
system we would start by putting all system management and job
initiation traffic on the Ethernet, and use the Myrinet for MPI
message.

-- 
Donald Becker				becker at scyld.com
Scyld Computing Corporation		http://www.scyld.com
410 Severn Ave. Suite 210		Second Generation Beowulf Clusters
Annapolis MD 21403			410-990-9993