cpu-rate 0.0.2 released, given proper URL, /proc/sysinfo request...

[Robert G. Brown]

Dear Beowulfers,

I've slicked up the cpu-rate tool I mentioned last week -- it is a bit
closer to ready for prime time.  It is also in a stage that will
facilitate its eventual integration with e.g. tools from the lmbench
suite.  I'm hoping to take the lmbench suite and add various things to
it (with the support/encouragement/active participation of that suite's
authors;-) to make an engineering support toolset for beowulfery and
general cluster design.

Because lmbench needs a fairly complete "host configuration file" in
order to function and because it doesn't hurt to have the information in
such a file in ANY decent system report, I've added a "config-host" tool
(written in perl) to the cpu-rate distro.  This tool automagically finds
out certain things about your system (at least it does if it is a linux
system not too different from mine, but I'm a linux kind of guy:-) and
writes them into an easily parsed config/hostname-config file.  This
configuration file still needs to be finished off by hand, because far
too many things that ought to be in it aren't readily extractable from
standard locations on a linux system (more on this later).

Once the host-specific config file exists and the application is built
(it is trivial and should just "build") in the cpu-rate directory, one
can run the "run-cpu-rate" script and generate in one step what should
be a statistically meaningful snapshot of the "optimum" single and
double precision arithmetic rates on your system (by optimum I mean data
in L1 cache).  Alternatively one can run cpu-rate itsef many times for a
variety of vector sizes and watch what happens to numerical performance
when a loop/vector size crosses a cache boundary.  I'll have a script to
drive this eventually.

I'd love comments on the output format of run-cpu-rate, the usability of
config-host, and cpu-rate itself.  Be sure to read carefully the remarks
in the README and/or man page -- it produces a "bogus" MFLOPS rating in
the same sense that bogomips are "bogus" MIPS, but even a bogus rating
may be useful to folks trying to engineer applications and clusters
(often at the same time).  I'd also love contributions of code or code
snippets and bug fixes.  At this instant it appears to work "perfectly"
on all the systems I have at my disposal to test, and indeed the results
and configuration files from those archetypical system are included in
tgz and src.rpm so they can be compared.

cpu-rate (0.0.2alpha or current snapshot) is available at

  http://www.phy.duke/edu/brahma

(click on the appropriate links to download tgz/i386.rpm/src.rpm as you
wish).  Larry and/or Carl, please look over the format of the host
configuration files and let me know what I'm missing that is needed to
drive lmbench.  Also note well the results/hostname.[1,2,3...N] files
generated by run-cpu-rate -n N.

Regarding system/host configuration information -- does anyone on the
list know if there are any plans to add e.g. /proc/sysinfo in 2.2.4?
Linux desperately needs a uniform and long-lasting source of system
configuration information as a lot of what there is comes from transient
log output from boot time or from when a module is inserted.  Obviously
this would greatly simplify MANY engineering tasks, including
system/kernel debugging and systems or beowulf engineering.  

At this point, it isn't possible to make any simple set of assumptions
about where to find obviously useful e.g.  /dev/eth0 or /dev/sd?
configuration information (and a lot of what one might want to know is
missing where there IS something logged someplace).  Linux may not need
a formal "device registry" that is used by the kernel, but a semi-formal
(standard format) "device registry" that is usable from userspace to
instantly tell you what your hard disk, network, CD drive, memory speed
and type, and so forth would be a great boon indeed.  Sort of like
/proc/cpuinfo but more complete and for other devices...  

The contents of config/hostname-config contains a lot of what I'd like
to see in such a file in a moderately extensible and easily parsed form
(run-cpu-rate contains an example of the perl variable/array parsing
code in the function "parseconfig()", although there might be some
benefit to converting to a hash-based/objectish format instead). It
would be very nice to have as much of the e.g.  manufacturer, make/model
number, size, speed, driver and so forth "registered" as possible in
something like a standardized, easily parsed format.  

Right now it isn't even possible to tell, as a general rule, what one's
ethernet device is and what speed and duplex it is running at or what
one's hard disks are from userspace if one's system doesn't "happen" to
be configured in such a way that they are in e.g.  /var/log/dmesg or
/var/log/messages and haven't been rotated out by log rotations and so
forth.  When one >>can<< access this information in a logfile, it is in
whatever format happened to suit the writer of the associated device
driver on that particular day.  A device registry of sorts would force
device driver writers to provide at least a small measure of
standardized information appropriate for their particular device class.

I also cannot believe that it would be THAT difficult a feature to add.
Similar things already exist for e.g. the pci bus, for example.  The
hardest part is likely agreeing on a common set of descriptor fields and
field lengths for given device classes, but even this cannot be THAT
difficult.

Just a wish list in case anybody from the kernel list is listening.  I
personally cannot mess with this right now, and understand that if I'm
not willing to do it myself I shouldn't hold my breath while waiting
for it...;-)

   rgb

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu