[Beowulf] single machine with 500 GB of RAM

[Vincent Diepeveen]

Hi,

What budget do you have and how fast must latency be to the RAM?

There will be a LOT of solutions to this.

If you can get a box with way more dimm slots that's always the ideal  
solution - as smaller dimms are way cheaper a gigabyte
than large dimms.

Sometimes you can glue together 2 motherboards. Bandwidth to the RAM  
still is ok then, yet random lookup latency total horrible.

So what really matters whether the software happens to stream (also  
some seemingly random patterns happen to systematically
eat RAM in a sequential manner like for example computer chess  
endgametablebase generation - so that's effectively also streaming),
or whether it looks up that RAM in really random manner (say search  
in Artificial Intelligence in all cases i'm aware of being optimized
well always works random to lookups in RAM). In case things are  
random lookup yet the number of bytes from each read is significant,
the overhead of random latency is less.

As for DDR2 versus DDR3, just look at price versus performance.  
Usually DDR3 is way faster.

Probably you'll end up with a 4 socket AMD machine with DDR3.

A single core is not so fast, but the bulldozer architecture might  
scale pretty well for the RAM latency,
because of how communication between CPU's work,
so reducing latency at the moment that you need to run a few cores,  
that dig in the same RAM randomly.


On Jan 9, 2013, at 1:42 PM, Jörg Saßmannshausen wrote:

> Dear all,
>
> Happy New Year!
>
> I was wondering whether people on the list here have some first hand
> experiences with this. I have been asked to purchase a single  
> machine with
> around 500 GB of RAM. We would not need more than 8 cores here. The  
> job simply
> needs that much of memory (and even then it is running for 14 days).
>
> Now, with that amount of memory used by a single core, I would have  
> thought
> that I need a fast memory interconnect, i.e. a high memory  
> bandwidth. I was
> thinking of getting an Intel Sandybridge CPU (maybe a  E5-2650) for  
> that
> machine and get a motherboard with can cope with that amount of  
> memory. Does
> anybody happen to have some recommendations here or knows of potential
> pitfalls?
>
> Also, in a related problem, how would I set the kernel.shmmni,  
> kernel.shmall
> and kernel.shmmax values so I am not running out of memory handles  
> here. I am
> still confused by that.
>
> Any kind of advice here is much appreciated. Given it is an  
> expensive piece of
> hardware we want to purchase I want to get it right.
>
> All the best from a grey London
>
> Jörg
>
> -- 
> *************************************************************
> Jörg Saßmannshausen
> University College London
> Department of Chemistry
> Gordon Street
> London
> WC1H 0AJ
>
> email: j.sassmannshausen at ucl.ac.uk
> web: http://sassy.formativ.net
>
> Please avoid sending me Word or PowerPoint attachments.
> See http://www.gnu.org/philosophy/no-word-attachments.html
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf