[Beowulf] slightly [OT] smp boxes
larry.stewart at sicortex.com
Sun Oct 19 19:20:01 PDT 2008
On Oct 19, 2008, at 5:15 PM, Mark Hahn wrote:
>> The SiCortex systems are clusters of 6-core SMPs. There is no load/
>> store access to memory on other nodes, although the interconnect is
>> fast enough to make software access to remote memory quite
> interesting way to put it, since competing interconnects are at
> least as fast (I'm thinking about the usual IB claim of ~1 us
I see HPCC results around 1.2 usec for Infinipath.. The submissions
at those sort of timings are pretty thin.
I am embarassed to say we haven't submitted a full run either.
>> I've been coding shmem and GASnet implementations for the SiCortex
>> interconnect recently, and on the older 500 MHz systems a "put"
>> takes about 800 ns and a "get" takes a little under 3 microseconds
>> before any particular
> so the 800ns put is half RTT for two nodes doing ping-pong puts?
No, It is bad scholarship. That figure is back to back puts, which is
I expect the latency to be similar to MPI Send+Recv which is around
1.4. I'll measure the
>> On the remote-paging side, I put together a prototype that gets
>> about 2 GB/sec and 64K page fault latencies under 100 microseconds,
>> again, not optimized.
> ouch. do you know how much of that time is due to slow MMU
> (at 2 GB/s, the wiretime for 64k should be 33 us, if I calculate
The 2 GB figure is aggregate bandwidth of multiple requests using
multiple rails. Any particular transfer uses only one rail (right now)
and the latency includes the server's not particularly polished
>> after all. I am kind of amused by all the press and angst about
> well, academics need _something_ to motivate grants, and the gov
> isn't likley to support all the nation's CS depts on gaming
> research ;)
> also, I suspect intel and perhaps sun have encouraged the brouhaha.
More information about the Beowulf