<font size=2 face="sans-serif">Mikhail,</font>
<br>
<br><font size=2 face="sans-serif">I still think that there could be a
NUMA issue here</font>
<br>
<br><font size=2 face="sans-serif">With no NUMA binding:</font>
<br><font size=2 face="sans-serif"> - the one process case can migrate
between cores on the core sockets - if its memory is on the first socket,
then it will run a little slower when scheduled on the second socket.</font>
<br><font size=2 face="sans-serif">- with two process on a node, the first
maybe be inhibited from moving to the other socket because there is already
a process there consuming cpu. and vice versa.</font>
<br><font size=2 face="sans-serif"> hence both will always run with
local memory.</font>
<br>
<br>
<br><font size=2 face="sans-serif">Daniel</font>
<br>
<br>
<br>
<br>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">"David Mathog"
<mathog@caltech.edu></font>
<br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">beowulf@beowulf.org</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">24/05/2011 19:27</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [Beowulf]
Execution time measurements</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">beowulf-bounces@beowulf.org</font>
<br>
<hr noshade>
<br>
<br>
<br><tt><font size=2>Another message from Mikhail Kuzminsky, who for some
reason or other <br>
cannot currently post directly to the list:<br>
<br>
BEGIN FORWARD<br>
<br>
1st of all, I should mention that the effect is observed only for<br>
Opteron 2350/OpenSuSE 10.3.<br>
Execution of the same job w/the same binaries on Nehalem E5520/OpenSuSe<br>
11.1 gives the same time for 1<br>
and 2 simultaneously runnung jobs.<br>
<br>
Mon, 23 May 2011 12:32:33 -0700 пиÑьмо от "David Mathog"<br>
<mathog@caltech.edu>:<br>
> Mon, 23 May 2011 09:40:13 -0700 ÿøÑÂьüþ þт<br>
"David Mathog"<br>
> <mathog@caltech.edu>:<br>
> > > On Fri, May 20, 2011 at 02:26:31PM -0400, Mark Hahn forwarded
a<br>
message:<br>
> > > > When I run 2 identical examples of the same batch job<br>
> > simultaneously, execution time of *each* job is<br>
> > > > LOWER than for single job run !<br>
<br>
> I thought also about cpus frequency variations, but I think that null<br>
output<br>
> of<br>
> lsmod|grep freq<br>
> is enough for fixed CPU frequency.<br>
><br>
> END FORWARD<br>
<br>
> Regarding the frequencies, better to use<br>
> cat /proc/cpuinfo | grep MHz<br>
<br>
I looked to cpuinfo, but only manually - some times (i.e. I didn't run<br>
any script w/periodical looking for CPU frequencies).<br>
All the frequencies of cores were fixed.<br>
<br>
> Did you verify that the results for each of the two simultaneous runs<br>
> are both correct? <br>
Yes, the results are the same. I looked also to number of iterations etc.<br>
But I'll check outputs again.<br>
<br>
>Ideally, tweak some parameter so they are slightly<br>
> different from each other.<br>
<br>
But I don't understand - if I change slightly some of input parameters,<br>
what may it give ?<br>
<br>
> David Mathog<br>
> mathog@caltech.edu<br>
> Manager, Sequence Analysis Facility, Biology Division, Caltech<br>
<br>
Fri, 20 May 2011 20:11:15 -0400 message from Serguei Patchkovskii<br>
<serguei.patchkovskii@gmail.com>:<br>
> Suse 10.3 is quite old; it uses a kernel which is less
than perfect<br>
at scheduling jobs and allocating resources for >NUMA systems. Try<br>
running your test job using:<br>
><br>
> numactl --cpunodebind=0 --membind=0 g98<br>
<br>
numactl w/all things bound to node 1 gives "big" execution
time ( 1 day<br>
4 hours; 2 simultaneous jobs run faster), for forcing different nodes<br>
for cpu and memory - execution time is even higher (+1 h). Therefore<br>
effect observed don't looks as result of numa allocations :-(<br>
<br>
Mikhail<br>
<br>
END FORWARD<br>
<br>
My point about the two different parameter sets on the jobs was to<br>
determine if the two were truly independent, or if they might not be<br>
interacting with each other through checkpoint files or shared memory,<br>
or the like.<br>
<br>
Regards,<br>
<br>
David Mathog<br>
mathog@caltech.edu<br>
Manager, Sequence Analysis Facility, Biology Division, Caltech<br>
_______________________________________________<br>
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe) visit </font></tt><a href=http://www.beowulf.org/mailman/listinfo/beowulf><tt><font size=2>http://www.beowulf.org/mailman/listinfo/beowulf</font></tt></a><tt><font size=2><br>
</font></tt>
<br>