rshbench 1.0.2

[Robert G. Brown]

OK, I made the changes suggested by the last few days' discussion.

rshbench 1.0.2 now uses /usr/bin/uptime as its test binary (although
this can easily be changed).  It has a loop function that allows all the
timings to be done with a very simple call.  It does all the requisite
postprocessing arithmetic to allow the average times to be isolated by
layer.  That is, it determines the average empty loop time, the average
time to execute /usr/bin/uptime, the average time required to execute
/bin/*sh -c /usr/bin/uptime, and the time required to execute [r,s]sh
targethost /usr/bin/uptime AND prints out a summary where the time of
the previous layer is subtracted out (to isolate the time required to
"execute an rsh" exclusive of a the underlying shell or command).  It
also extracts the cpu name and speed and prints it, the command being
tested, and the shells being tested in the output header.

It is distributed as rpm, src.rpm and .tgz from

  http://www.phy.duke.edu/brahma 

as previously noted.

I've done a lot more testing and (in addition to improving the benchmark
numbers presented in the README of the distribution) and can give the
following approximate summary (approximate because times vary somewhat
with CPU speed).
             time (seconds)
rsh          0.09
ssh          0.25
openssh+pam  0.75  (pam_pwdb nodelay!)
ssh+pam      0.8   (pam_pwdb nodelay!)
ssh+pam      1.76  (pam_pwdb allowing delay)
rcp 1k       0.11
rcp 1M       0.20
scp 1k       0.27  (default -c idea)
scp 1M       0.67  (default -c idea)
scp 1k       0.27  (-c none)
scp 1M       0.38  (-c none)

(where one typically has to configure --with-none in the ssh build to
enable the latter, in case you wonder why ssh -c none doesn't work at
your site).  These are perfectly understandable.  It takes approximately
0.1 seconds to send an unencrypted 1 MB file on a 100BT connection for
both rsh and ssh, but ssh still has the extra 0.16 or so seconds
required to authenticate the connection.  I don't show pam times because
they are layered on top of that.  I don't believe that NIS delays are
relevant because when I test e.g.

  ypcat passwd | grep rgb | awk -F: '{print $2}'

which is a total hog userspace way of getting a passwd entry, it only
takes 0.07.  I cannot believe that any possible combination of systems
calls run through NIS are going to be worse than an abomination like
this run from a shell, or if you prefer, I'd guess NIS per se
contributes strictly less than 0.1 seconds to the pam contributions
above.  I also played games with nsswitch.conf and adding flatfile
entries to avoid NIS and saw no significant difference in the times.

Conclusion:  bproc obviously wins hands down but requires kernel
modification which isn't for everybody or all kinds of clusters -- it is
a "true beowulf" solution.  The marginal cost of running ssh instead of
rsh is roughly 0.16 seconds per transaction, plus whatever encryption
burden you prefer or require.  pam-ified ssh is evil (for whatever
reason) -- it appears that pam should really stand for Porcine
Authentication Module as far as its speed is concerned.  However, I
haven't gone deep into the bowels of pam (so to speak:-) to identify
just where these shameful delays are occurring, so it may be a
misconfiguration error and not pam at all.  I would definitely avoid
using ssh on top of pam, though, unless you absolutely must or unless
you have time to identify and fix the problem or unless you're using it
to manage only a few (hundred) transactions per day and can afford the
minute or two of CPU this will cost.

I'd still love to hear from anyone with further benchmarks or
observations, but the numbers above are probably fairly accurate rules
of thumb for the time being.

    rgb

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu