[Beowulf] Time measurement

[Robert G. Brown]

Vincent Diepeveen writes:

>>these microbenchmark contexts it is actually a PITA to "prepare" the
>>system in such a way as to make interruptions like this unlikely in the
>>timing interval(s) and why writing a "good" microbenchmark program is
>>nontrivial.
> 
> As soon as you allow scientists to do measurements of their results
> without wall time clock, then problems really will grow above Mount Everest.
> 
> The stopwatch is what counts!

Vincent, I don't know what kind of distorted world view you have of
"scientists" but as a card-carrying physics Ph.D. scientist who works
with scientists, teaches science, -- that's just plain silly.

First of all, nobody "allows" scientists to do measurements in any
particular way or no way at all.  They think about the problem (ANY
problem) and based on their understanding and research decide if and/or
how to do a measurement, if a measurement is relevant to its solution.
I honestly don't think you could find a single name-identifiable group
on the planet that is LESS likely to need "permission" to make
measurements in order to learn useful things about ANY problem than
scientists.

Second, if "the stopwatch is what counts" then when I run a
stopwatch-based timing of my job at a time that another user happens to
ALSO be running a job, well, my measurement of twice (or more) as long a
time required to run "counts".  Or if I run my job through a batch job
scheduler (as was recently pointed out on list) and it happens to get
swapped out so something else in the queue can play through, don't worry
-- just quote the wall clock time as being THE accurate measurement of
how long it takes the job to run, the one that counts.  Not.

Third, as a scientist (!) who has been working on microbenchmark
programs for close to two decades now, with varying degrees of accuracy
and improvements over the years, please allow me to reiterate that in a
microbenchmark program with an actual timing loop, what counts is the
most accurate clock you can find on the system (the CPU cycle counter in
most modern architectures).  However, it is not used over the ENTIRE
timing applications -- it is used over the particular code fragment you
are trying to time, ideally many times over for decent statistics over
that pesky old system state.  Not really "wall clock time" -- a better
analogy is a stopwatch used to time many races, where the time of day
the race is run is irrelevant.  You see, scientists are actually quite
capable of understanding the nuances of timing computer programs and
while the problems DO tend to mount up, it isn't because they are
scientists it is because timing anything on a computer is a thorny
problem with a Heisenberg-like element to it; measuring the timing can
easily CHANGE the timing.

Fourth, if you want to PROFILE an application to learn where it is
spending the most time, you have two ways (AFAIK) to proceed.  You can
compile the application with debugging/profiling turned on, in which
case the compiler will instrument the code for you in such a way that
gprof (for example) can determine/report the internal timings of all the
separate calls and program modules, or you can instrument the code
yourself with the aforementioned stopwatch (cycle counter,
gettimeofday).  In the former case wall clock time is NOT relevant at
all.  Everybody knows the instrumented code will not run as efficiently
as stripped code (Heisenberg strikes, sort of:-), but the RELATIVE
timings are still useful and approximately correct information.  In the
latter case wall clock time is STILL irrelevant, even if you are using a
"wall clock timer" like gettimeofday as a stopwatch.

I hope that it is pretty obvious that most "scientists" can cope with
the massive complexities of using gcc -g -p instead of gcc (ALL OTHER
THINGS being utterly equal), and that running gprof is straightforward
and its results self explanatory.  It is also well-documented in any
number of places as a standard programming tool that even non-scientists
might benefit from using, sometimes, if they too can get "permission";-)

So not only is your implication that scientists are somehow brain-dead
individuals who have to be protected from themselves around the
computers they bought and in most cases actually hand programmed
themselves to accomplish their research both demeaning and incorrect,
your general CONCLUSION is incorrect as well, even for NON-scientists,
mathematicians, programmers, or chess enthusiasts.  Anybody interested
in timing a program, optimizing a program, understanding a program in
any sort of detail at all is bound to find profiling useful, and most
professional or semi-professional programmers (including lots of those
"scientists") use it.  Profiling does not involve wall clock time in any
relevant way. 

QED.

  Sincerely,

      rgb
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20050808/605152e4/attachment.sig>