[Beowulf] Altix vs. Beowulf

[Mark Hahn]

> My experience is that when altix has numalink3 the latencies even at 64
> processors are, as you can see in the report of prof Aad v/d Steen who
> tested it for dutch government, about 3-4 us.

should be in the 1-2 us range.  no doubt it does depend on size,
numalink version (3/4), topology, processor, etc.

> Any cluster gives that too for a far smaller price.

"any" is quite wrong, of course (eg gigabit).  but there are numerous
interconnects in that latency range.

> The only advantage of altix is that it has a shared memory model. 

being one integrated machine is certainly an advantage, as well,
in terms of maintainability, administration, etc.  that said, I'd love
to see numalink4 cards that plug into coherent HyperTransport ;)

> So if you use shared memory up to a cpu or 16, it's a fine machine.
> 
> Above that i definitely would consider getting a cluster with a good network.

this is an odd statement - implying that shared memory doesn't scale above
16x.  that's not true in general, of course.

> Please note that several network cards also have build in shared memory at
> the card. Quadrics has a shmem library for example at a real low cost for 8
> nodes.

a shmem library does not shared memory make.

> If you buy for $13k a quadrics 8 node network, and 8 quad opterons for $9000,
> then for far under $100k you have yourself a 32 processor machine, which
> you can very cheaply update to dual core in start 2006 too, so you always

make that "to dual core tomorrow" (though perhaps not "very cheaply" ;)

> For real fast latency between only a very few nodes, Dolphin seems to have
> an excellent network.

mainly an issue of topology, not number.