Beowulf: A theorical approach

[Greg Lindahl]

> > But it isn't actually cache coherent. Remember Cray shmem(): To use
> > data, you fetch it to be close to you, and then it isn't cache
> > coherent while it's local. It IS cache coherent in that the fetch
> > gets you the right data.
>
> Wrong. In the CS2 it _was_ cache coherent both locally and remotely.

I wasn't discussing the CS2, I was discussing the T3E and SALC. Sorry if I
gave a different impression. I don't know of any other machines like the
CS2, nor do I think they're interesting.

> Another of the reasons I don't like the SALC description is precisely
> this confusion about what the "local consistency" is intended to mean.

You can take it up with Bob Numrich; I'm just the messenger. Bob invented
the shmem interface in the first place. shmem is interesting mainly because
it's cheaper to build hardware for it than for things like the CS2.

> Making the NIC properly cache coherent is one of the main reasons to
> be on the processor bus, appearing as a second CPU. It allows the NIC
> to implement the full coherency protocol when accessing data (either
> bringing it in, or sending it out).

That's far more expensive and difficult than the other benefit of getting on
the processor bus: reduced latency.

-- g