Robert G. Brown rgb at phy.duke.edu
Thu Oct 11 11:39:21 PDT 2001

On Thu, 11 Oct 2001, Richard C Ferri wrote:

> Having followed the NIS discussion, I have a similar question about setting
> the date on the compute nodes. Are most people:
> using NTP on every compute nodes?

Hi Rich,

We've used ntp for time sync on both nodes and LAN workstations for a
decade or so.  Reasons:

  a) If you have a good synchronization tier (of "strata") set up, it is
very, very accurate.  Accuracy of less than a second is standard, often
to within a few milliseconds.  On all clients.  It actually corrects for
network transit times of the time synchronization packets dynamically
and also dynamically "adjusts" your clock after measuring its average
rate of drift.
  b) Consequently your hosts/nodes remain so synchronized that you never
see any of the annoying messages from make or other systems utilities
warning you about files with modification dates in the future and
incomplete builds or possible attacks and countermeasures.  Log messages
sent to a common loghost are all referenced by a common time standard as
  c) ntp has been around in one form or another for many years, is
highly stable, and works invisibly and well.
  d) Without ntp (or some form of synchronization) at least some
motherboards get WAY out of sync very quickly.  I've seen drift rates of
tens of seconds a day.  This is one reason to use a daemon that
establishes absolute time from a certified national clock site (via a
suitable chain of stratum N servers, with N likely >3 or 4 at the client
level, so you don't whack the master clocks with too much traffic). Just
sync'ing to a master node in your cluster won't guarantee that it is
sync'd to true time.  Rather all your cluster nodes could be drifting
around with the master node.

The only negative side I've ever seen to it is that it used to die
without much of a complaint if it ran into trouble, and one form of
trouble it would run into was that would fail if the clock was offset by
too large an interval from the time it was trying to set.  If the
hardware clock wasn't reset for any reason, it would often fail at boot
time and you'd only learn about it if you checked or when a problem
surfaced.  At the very least, you had to set the date/time by hand to be
approximately right before cranking up ntp to refine it.

It LOOKS like the latest couple of versions of ntp in the RH 7.x series
have been smart enough to eliminate all or most of these problems -- RH
resets the hardware clock automatically (during boot or shutdown, don't
remember which) and ntp doesn't seem to fail very much if at all any
more, silently or otherwise, no matter what the clock looks like at
boot.  This is very fortunate, as we tend to install nodes and
workstation clients via unattended, floppy-free kickstart, and don't
want to have to come back and "fix" time synchronization problems by

BTW, procstatd returns system time as one of the monitored fields, as I
found that it was a good way to detect ntp failures and fix things
before log records and the like got screwed up.  It was always very
interesting to watch a new system's clock "gradually" get pulled into
sync with all the other nodes by ntp so that they all read exactly the
same thing.


Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu

More information about the Beowulf mailing list