[Beowulf] single machine with 500 GB of RAM

[Joe Landman]

On 01/09/2013 07:42 AM, Jörg Saßmannshausen wrote:
> Dear all,
>
> Happy New Year!
>
> I was wondering whether people on the list here have some first hand
> experiences with this. I have been asked to purchase a single machine with
> around 500 GB of RAM. We would not need more than 8 cores here. The job simply
> needs that much of memory (and even then it is running for 14 days).

We've built/delivered/supported machines with 1+TB before.  They aren't 
too uncommon these days, and you have a number of choices w.r.t. them.

If you can share more about the specific nature of the calculation, 
you'd likely get better recommendations.  That is, large memory, 
light/no threading could be a very large matlab job (we've seen this) 
among other things.  This would impact CPU choice as well as RAM speed 
choices.

And this gets to a more design focused discussion as well.  You can 
build a single machine with this much ram, or aggregate multiple 
machines with vSMP from ScaleMP and use somewhat less expensive RAM.

>
> Now, with that amount of memory used by a single core, I would have thought
> that I need a fast memory interconnect, i.e. a high memory bandwidth. I was
> thinking of getting an Intel Sandybridge CPU (maybe a  E5-2650) for that
> machine and get a motherboard with can cope with that amount of memory. Does
> anybody happen to have some recommendations here or knows of potential
> pitfalls?

Don't design the system too early without a better discussion of the 
process.  Apart from being memory bound, are you CPU bound?  IO bound? 
Network bound?

>
> Also, in a related problem, how would I set the kernel.shmmni, kernel.shmall
> and kernel.shmmax values so I am not running out of memory handles here. I am
> still confused by that.
>
> Any kind of advice here is much appreciated. Given it is an expensive piece of
> hardware we want to purchase I want to get it right.
>
> All the best from a grey London
>
> Jörg
>


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615