[Beowulf] slightly [OT] smp boxes

[Lawrence Stewart]

On Oct 17, 2008, at 10:06 PM, Joe Landman wrote:

>>
>> The Sicortex system has lots of cores and some of their smaller
>> development systems and might qualify if the goal is to research
>> SMP, NUMA, and single system memory image.
>
> I don't know if you would call them SSI, and I don't remember if  
> they have strict coherency within them.  This said, they have a very  
> fast and scalable fabric.  It is an interesting system.  Not a  
> commodity box, but quite interesting.

The SiCortex systems are clusters of 6-core SMPs.  There is no load/ 
store access to memory on other nodes, although the interconnect is  
fast enough to make software access to remote memory quite interesting.

All the nodes typically run the same kernel image, but it isn't an SSI  
system today.

I've been coding shmem and GASnet implementations for the SiCortex  
interconnect recently, and on the older 500 MHz systems a "put" takes  
about 800 ns and a "get" takes a little under  3 microseconds before  
any particular effort on performance.  These are in line with MPI   
PingPong  times.  As far as I can tell, these will scale with clock  
speed on the 700 MHz systems.

On the remote-paging side, I put together a prototype that gets about  
2 GB/sec and 64K page fault latencies under 100 microseconds, again,  
not optimized.

I'm kind of hoping some academic departments use the iron for  
operating systems, programming,  and communications research.  It's  
all open source after all.  I am kind of amused by all the press and  
angst about "multicore", like 2, 4, or 8 cores are even an interesting  
problem.  It seems to me that 100s of cores are pretty well  
understood, but when you get to 1000's, there is a lot more to learn.   
Of course I had a 5 cpu workstation back in 1986 or so...

-Larry/SiCortex