[Beowulf] precise synchronization of system clocks

[Lux, James P]

On 9/30/08 12:01 PM, "Donald Becker" <becker at scyld.com> wrote:

> On Tue, 30 Sep 2008, Lawrence Stewart wrote:
>
>> On Sep 30, 2008, at 12:07 PM, Lux, James P wrote:
>>> On 9/30/08 8:37 AM, "Robert G. Brown" <rgb at phy.duke.edu> wrote:
>>> The grottiest grungy GPS receiver can probably do 100ns on its 1pps
>>> tick,
>>> and most are in the 20ns range.  There ARE receivers that have
>>> systematic
>>> errors (i.e. Some sort of sawtooth in the error) and, of course,
>>> there are
>>> countless schemes to compensate in one way or another.
>> So someone with some grad students and soldering irons should hook up
>> the 1 pps
>> output of a GPS to, say, the carrier detect pin on the unused RS232
>> ports of all their
>> nodes, and write some software...
>
> Note that you need a "component" GPS receiver, hooked up with a bunch of
> wires.  The 1 PPS output isn't available from your typical packaged
> handheld receiver.  Those output serial NMEA lines that might be a full
> second out of date.

On some receivers, the actual message is also synchronized with the "tick",
as in there's a deterministic relationship between when a particular
character of the message starts/ends and the 1pps hack.  If you have a
non-FIFO UART, you could conceivably synchronize off the "Receive Character
Ready" status.


>
> And once you think about dealing with a 1 PPS signal, you find that you
> don't even need a GPS.  Just a pulse generator.

Indeed, this is true.  For the things we've been discussing here (other than
demonstrations of GR), what you really need is just synchronization within
the cluster, not synchronization of the cluster to some outside reference.

Oh, and for those of you contemplating syncing those nodes up to
nanoseconds, bear in mind that the absolute frequency accuracy of those
clocks is probably in the 100 ppm range, so in a second, you could already
have drifted 10 milliseconds.  Programs like ntpd DO model this behavior,
but if you're doing barrier syncs (or trying to get your tasks to all do the
same thing at the same time) you'll probably need to deal with it.

  And you can do some other
> clever things.  Or just skip ahead and read about Purdue PAPERS -- figure
> out why it was very appealing but failed.
>
> Meanwhile, I'll try to find out where I can plug a serial cable into a
> modern server...
Simple.. It hooks up to that 25 pin male connector on the back of the
Asynchronous Communications Adapter that plugs into the ISA bus next to your
memory expansion card so you can have more than 64K of RWM. Unless you want
to cobble something that hooks up to the cassette i/o port, which I think is
a 5 pin DIN.  You could also use one of the lines on the printer port of
your Monochrome Display Adapter. Several of those have IRQs hanging off
them, and it's easier coding than the 8250.