[Beowulf] X5500

[Vincent Diepeveen]

Bill,

the ONLY price that matters is that of ECC ram when posting in a  
cluster group.


If there is 1 commission that EVER puts a signature underneath a  
production cluster
without ECC ram using x86 processors (gpu's is yet another new thing  
that is interesting
to discuss), then please inform me, as they qualify for a full and  
thorough investigation
by a range of shrinks and psychologists, on how group behaviour could  
lead to such a
total unqualified and naive and total wrong decision; resulting of  
course in the direct
firing of the entire commission and decommissioning them to north  
part of
Norway where they can count the number of iceblocks they see afloat,  
this for the rest of
their life until retirement age,.

So in short i can completely ignore your posting.

ECC is a requirement, not a luxury.

On Apr 3, 2009, at 5:04 AM, Bill Broadley wrote:

> Vincent Diepeveen wrote:
>> I wouldn't bet at registered-ecc DDR3 ram to become cheaper.
>> To be honest i misjudged that for DDR reg-ecc ram also,
>> it still is relative spoken expensive.
>
> I've heard this a dozen times, seems repeated quite often.  Yet when I
> actually look I see it either so small as to be inconsequential or  
> even negative.
>
> 6GB ram for a dual nehalem system:
> http://www.newegg.com/Product/ProductList.aspx? 
> Submit=ENE&DEPA=0&Order=BESTMATCH&Description=DDR3-1066+ECC&x=0&y=0
> $96.99 / 6 = $16.165 per GB
>
> 4 GB ram
> http://www.newegg.com/Product/ProductList.aspx? 
> Submit=ENE&DEPA=0&Order=BESTMATCH&Description=2GB+DDR2-800+ECC&x=0&y=0
> $109.99 / 4 = $27.49 per GB
>
> Sure the later is fully buffered and the former is not, but  
> registered isn't
> required for up to 6 dimms in a dual socket.
>
> At silicon mechanics a dual nehalem upgrade from 12GB of DDR-1333 - 
> > 24GB of
> (swapping 2GB dimms for 4GB
> DDR-1333 = $348 / 12 = $29 per GB.
>
> For an opteron system to swap 2GB dimms for 4 = $760 /16 = $47.50  
> per GB.
>
> Where is the horror of the high DDR3 prices?
>
>> DDR2 ecc ram on other hand is so dirt cheap,
>
> Previous comparisons for desktops showed about a $25 difference for  
> your
> average new highend desktop with 4-8GB ram.  Hardly something worth  
> noting IMO.
>
>> that you really can make nodes with a lot of ram really cheap.
>
> Indeed, a dual xeon E5530 with 24GB ram is around $3k, which I  
> think is very
> reasonable.
>
>> Both amd as well as intel have boards where you can put in easily  
>> 64GB ram,
>> some even you could go to 128GB ram.
>
> Indeed, the above mentioned xeon E5530 with 48GB ram is $3700.
>
>> Latency of DDR3 is not really good cmopared to DDR2. Of course this
>> depends upon how you use
>> the RAM.
>
> Not really good?  I've seen a decent improvement using the normal,  
> my plat, or
> lmbenchs.
>
>> That i7 can deliver the weird amount of 192 bytes at a time or so,
>> versus dual channel ddr2 can deliver 64 at a time.
>
> Er, well the newer AMD chips can return 64 at a time on 2 channels,  
> and the
> new intel is 64 at a time from 3.  Are you arguing aesthetics or  
> performance?
>  Considering that often the intel get's a fair bit more performance  
> with 8
> threads vs 4 it seems pretty reasonable to add another channel to  
> maintain the
> performance vs memory bandwidth or IMO more important performance  
> per random
> memory reference.
>
>> Now that's a huge bandwidth difference of course, some i7's even  
>> get to
>> 18GB/s bandwidth, versus 10GB/s for the DDR2
>
> I've seen 22GB/sec on a single socket, it did require using  
> pathscale's
> compiler.  The dual sockets are of course better... unlikely the  
> previous
> generation dual socket intels which were often SLOWER than the  
> desktops.
>
>> equivalent; yet that means that the latency of the DDR2 quadcores is
>> simply better. More bytes at a time comes at a big latency
>
> I don't follow, do you have numbers?
>
>> price simply.
>>
>> If you're just having software that streams and hardly has  
>> something to
>> do for the cpu's to calculate while you stream,
>> then maybe consider rewriting the algorithm to something more complex
>> that needs to stream less and can do more calculations,
>> as doing massive calculations at just a few gigabytes of RAM is what
>> gpu's are genius at and will scale perfectly for, maybe
>> even outperforming the law of more there for a short period of  
>> time in
>> number of instructions a cycle you can push through in total
>> at a single gpu node.
>
> Certainly performance vs the previous generation varies, but it  
> seems that
> even cache friendly stuff is often 5-10% faster and that's before  
> you consider
> running 8 threads instead of 4.  If you are doing random lookups or  
> anything
> bandwidth intensive it's even better.  Once you look at dual  
> sockets (xeon
> 55xx) vs intel the difference is even bigger.
>
> So when you look at the performance, the price, and the power usage  
> it's a
> pretty compelling improvement vs the previous intel.  I'm not sure  
> if the AMD
> systems have price adjusted as a result yet but I expect them to.   
> Now it's
> AMDs turn, maybe they will ship some CPUs that take more than 75  
> watts now
> that Intel seems to have finally shipped something competitive.
>