[Beowulf] GPFS on Linux (x86)

[Greg Lindahl]

On Thu, Sep 14, 2006 at 08:08:12AM -0400, Scott Atchley wrote:

> If, however, you are considering something faster than gigabit  
> Ethernet, bear in mind that GPFS only can use TCP (as far as I know).  

The July 2006 presentation posted to the list about the same time as
your email says they're adding IB protocols in the future. I thought
that GPFS used to support GM directly?

> You will be limited to the performance of the TCP on your  
> interconnect. If you are using TCP/IP/Ethernet, then expect a very  
> high CPU load.

This is debatable. It is often the case that a large cluster does big
I/O (say 10+ GB/s) on so many nodes (1000+) that individual nodes are
doing < 100 MB/s. In that case the TCP overhead is not that large.

And frequently you're limited by the disk system, not the network.

But this depends, of course.

-- greg