[Beowulf] Advice on 4 CPU node configurations

[xingqiu yuan]

On Wed, Feb 9, 2011 at 2:37 PM, Prentice Bisbal <prentice at ias.edu> wrote:

> Craig Tierney wrote:
> > On 2/9/11 7:38 AM, Prentice Bisbal wrote:
> >> I don't think Numascale/ScaleMP has much of a cost advantage anymore.
> >>
> >> About 6 months ago, I purchased a couple of Dell PowerEdge R815s with
> >> 128 GB of RAM and 32 cores. We looked at similar RAM configurations a
> >> couple of years ago. and the cost premium for that much RAM was
> >> prohibitive (the price for additional RAM seemed to go up exponentially
> >> with the amount of RAM) so we stayed at 32 GB.  This time around, the
> >> premium for the additional RAM seemed marginal. Not sure how large you
> >> can go and keep that "marginal" relationship, but it's much larger than
> >> it was a couple of years ago.
> >>
> >
> > There is only so much memory you can put in a box, regardless of the
> > cost.  ScaleMP/Numascale let you get around that issue.  What if I need
> > a TB of RAM?  Yes, I might be able to convert my algorithms, but if I can
> > add $3k per node to get cache-coherent memory, why not go that route?
>
> >Agreed. I was going to add the caveat that there are limits to how much
> >RAM you can add to a single box, or there eventually be a point the
> >amount of RAM in a single box vs. price is no longer nonlinear, but then
> >I got lazy. ;)
>


Don't understand why you need TB of RAM? Don't forget the fact that $3K can
almost buy one-node!!!



> >
> >
> >> And don't forget to figure in the cost of the interconnect. I don't know
> >> what Numascale requires, but ScaleMP require IB, which can add
> >> significant costs if you don't have an existing IB fabric to use.
> >>
> >
> > ScaleMP is a software solution, Numascale is a cache-coherent hardware
> solution.
> > The list cost of the 3D torus version Numascale card with 4GB Ram
> (highest end card).
> > Cables are similar to IB, about $120 each.
> >
> > It would be really cool to build SMP system to sit next to my cluster.
>  When users
> > need large memory, or want to do things with UPC or other similar
> languages,
> > an SMP system like this would be handy.
> >
> > - I am not advocating for Numascale, or any other products here.  I am
> just
> > trying to provide information to the community.
> >
> > Craig
> >
> >
> >
> >> Prentice
> >>
> >>
> >> Douglas Eadline wrote:
> >>> Although I wrote about the SMP and the Numascale hardware,
> >>> I have not yet had a chance to use it.
> >>>
> >>> To me there are two issues worth considering. First,
> >>> if you need a big SMP this might be a low
> >>> cost solution. Second, should your application
> >>> not scale best in a node-based ccNuma system (ScaleMP
> >>> or Numascale), MPI is still an option. Indeed, no need
> >>> to rewrite the codes. And, management is probably
> >>> much easier.
> >>>
> >>> Of course for large systems, clusters work best.
> >>>
> >>> --
> >>> Doug
> >>>
> >>>
> >>>> On Thu, 2011-02-03 at 09:45 +0000, Hearns, John wrote:
> >>>>> Also look at ScaleMP and Numascale
> >>>>> Here's a damn good article from Doug Eadline:
> >>>>> http://www.linux-mag.com/id/7947
> >>>>>
> >>>>> I must admit though I don't know how far a budget of 30K takes you
> >>>>> there!
> >>>> My understanding is that the Numascale cards cost "a bit more" than IB
> >>>> cards but you don't need a switch (it's a 3D taurus I think) so it
> would
> >>>> be worth looking into.
> >>>>
> >>>> Rob
> >>>>
> >> _______________________________________________
> >> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
> Computing
> >> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> > To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
> >
>
> --
> Prentice Bisbal
> Linux Software Support Specialist/System Administrator
> School of Natural Sciences
> Institute for Advanced Study
> Princeton, NJ
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20110209/4b10b8d0/attachment.html>