[Beowulf] Re: dual core (latency)

[Vincent Diepeveen]

At 11:05 AM 7/19/2005 +0800, Stuart Midgley wrote:
>The first thing to note is that as you add cpu's the cost of the  
>cache snooping goes up dramatically.  The latency of a 4 cpu (single  
>core) opteron system is (if my memory serves me correctly) around  
>120ns.  Which is significantly higher than the latency of a dual  
>processor system (I think it scales roughly as O(n^2) where n is the  
>number of cpu's).
>
>Now, with a dual core system, you are effectively halving the  
>bandwidth/cpu over the hyper transport AND increasing the cpu count,  
>thus increasing the amount of cache snooping required.  The end  
>result is drastically blown-out latencies.
>
>Stu.

This doesn't answer even remotely accurate things.

A) my test is doing no WRITES, just READS.
B) snooping might be for free.
C) all other cores are just idle when such a latency test for just 1 core
happens and the rest of the system is idle.
D) in all cases a dual core processor has a SLOWER latency and it doesn't
make sense.
E) you don't seem to grasp the difference between LATENCY and BANDWIDTH;

For example your BANDWIDTH to Mars might be GREAT, but your LATENCY to Mars
is real ugly, as it takes 200 years for them to return. 

You keep mixing latency and bandwidth. That's ugly, to say polite.

I'm speaking of LATENCY here, not bandwidth.

The total BANDWIDTH that my program takes at a dual core is to be correct:

8 bytes * 1 billion (1/ns) / 147 (ns) = 54MB/s

In fact with some luck your gigabit ethernet card might be able to handle
54MB/s.

Vincent

>
>On 19/07/2005, at 10:50, Vincent Diepeveen wrote:
>
>> Hello Stuart,
>>
>> Thanks for your answer regarding numactl tools.
>>
>> Your answer doesn't necessarily explain why the dual core latency  
>> (with or
>> without numactl) is far worse, yes 30%+ worse, than that of single cpu
>> opterons of the same speed, when benchmarking just 1 core (so the  
>> others
>> sitting idle).
>>
>> Any thoughts on that?
>>
>> Thanks,
>> Vincent
>>
>
>
>--
>Dr Stuart Midgley
>Industry Uptake Program Leader
>iVEC, 'The hub of advanced computing in Western Australia'
>26 Dick Perry Avenue, Technology Park
>Kensington WA 6151
>Australia
>
>Phone: +61 8 6436 8545
>Fax: +61 8 6436 8555
>Email: industry at ivec.org
>WWW:  http://www.ivec.org
>
>
>
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org
>To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
>
>