[Beowulf] Advice on 4 CPU node configurations

[Craig Tierney]

On 2/9/11 7:38 AM, Prentice Bisbal wrote:
> I don't think Numascale/ScaleMP has much of a cost advantage anymore.
>
> About 6 months ago, I purchased a couple of Dell PowerEdge R815s with
> 128 GB of RAM and 32 cores. We looked at similar RAM configurations a
> couple of years ago. and the cost premium for that much RAM was
> prohibitive (the price for additional RAM seemed to go up exponentially
> with the amount of RAM) so we stayed at 32 GB.  This time around, the
> premium for the additional RAM seemed marginal. Not sure how large you
> can go and keep that "marginal" relationship, but it's much larger than
> it was a couple of years ago.
>

There is only so much memory you can put in a box, regardless of the
cost.  ScaleMP/Numascale let you get around that issue.  What if I need
a TB of RAM?  Yes, I might be able to convert my algorithms, but if I can
add $3k per node to get cache-coherent memory, why not go that route?


> And don't forget to figure in the cost of the interconnect. I don't know
> what Numascale requires, but ScaleMP require IB, which can add
> significant costs if you don't have an existing IB fabric to use.
>

ScaleMP is a software solution, Numascale is a cache-coherent hardware solution.
The list cost of the 3D torus version Numascale card with 4GB Ram (highest end card).
Cables are similar to IB, about $120 each.

It would be really cool to build SMP system to sit next to my cluster.  When users
need large memory, or want to do things with UPC or other similar languages,
an SMP system like this would be handy.

- I am not advocating for Numascale, or any other products here.  I am just
trying to provide information to the community.

Craig



> Prentice
>
>
> Douglas Eadline wrote:
>> Although I wrote about the SMP and the Numascale hardware,
>> I have not yet had a chance to use it.
>>
>> To me there are two issues worth considering. First,
>> if you need a big SMP this might be a low
>> cost solution. Second, should your application
>> not scale best in a node-based ccNuma system (ScaleMP
>> or Numascale), MPI is still an option. Indeed, no need
>> to rewrite the codes. And, management is probably
>> much easier.
>>
>> Of course for large systems, clusters work best.
>>
>> --
>> Doug
>>
>>
>>> On Thu, 2011-02-03 at 09:45 +0000, Hearns, John wrote:
>>>> Also look at ScaleMP and Numascale
>>>> Here's a damn good article from Doug Eadline:
>>>> http://www.linux-mag.com/id/7947
>>>>
>>>> I must admit though I don't know how far a budget of 30K takes you
>>>> there!
>>> My understanding is that the Numascale cards cost "a bit more" than IB
>>> cards but you don't need a switch (it's a 3D taurus I think) so it would
>>> be worth looking into.
>>>
>>> Rob
>>>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf