[Beowulf] Re: Finally, a solution for the 64 core 4TB RAM market

[Mark Hahn]

>> I would guess that most people who currently have clusters would rather get 
>> bigger/faster/cooler clusters, rather than go to SMP, unless for some
>> reason they have a fixed problem size.  possible, I guess.
>
> We intentionally built one cluster recently as a throughput system, with slow 
> (ok, gigabit) interconnect, while the latest is "HPC" with DDR IB 
> interconnect.
>
> We have throughput users (most jobs run on a single node, and can take 
> advantage of the node's memory footprint).  A number of these are SMP or 
> SMP-lite.  Did I mention computational chemistry?

sure - we have >3k users and lots of all these categories as well
(and have also specialized our clusters).  but the point is that 
8-socket fat nodes are going to be more expensive; traditionally 
nonlinearly more expensive.  current 4s boxes are more than 2x 2s cost.

having fewer nodes is also a value, but mainly only if you wind up with
single-digit numbers of nodes - if you're wrangling a cluster, it hardly
matters whether it's 200 or 400 nodes.  (again, fat nodes have not 
historically saved on power or space - at least not proportionally.)

> We also have some folk interested in map-reduce, but I've not been able to 
> accommodate them just yet.

yes, us too.  what are your thoughts on the kind of config that would suit
them - just the google sort of layout?  (gigabit, I suppose.  probably 
dual-socket, with however many 2G dimms will fit, and a couple large 
local disks)

> Depends on your mix of users.

I still think the market for 64c machines is relatively small.