Bob Drzyzgula bob at drzyzgula.org
Thu Oct 11 18:41:04 PDT 2001

On Thu, Oct 11, 2001 at 06:26:04PM -0400, Robert G. Brown wrote:
> On Thu, 11 Oct 2001, Greg Lindahl wrote:
> > Now if you want GOOD time synchronization (to within a microsecond)
> > that's a *much* harder problem...

> Yes, it starts to become very difficult and expensive right in there
> somewhere.  The so called "atomic clocks" one can now buy in stores like
> Wal Mart for $50 or so are autosync'd to WWVB at NIST in Colorado.
> Unfortunately, Colorado is about 1000 miles away, which is around 3
> microseconds right off the top that is difficult to compensate for.
> Also, this station broadcasts at only 60 kHz which means that time
> resolution is likely to be more like milliseconds or worse. Network
> jitter and so forth complicate net-based time sync connections, although
> one can get to millisecond accuracy on a good day (or over several good
> days).

Well, yes, but then again you can receive the signal inside a building.
GPS transmits at about 1.57GHz and the signal has trouble making it
even through trees. 

Also, WWVB is an excellent frequency source, with a transmitted uncertainty
better than one in 10^-11. And if you calibrate the path delay -- which is
reasonably stable because it tends to travel along the ground -- you
can get the time uncertainty down to 100us or so with a good receiver.

>    http://www.gpsclock.com
> offers a clock that is good (according to their spec) "to a millionth of

I've used TrueTime -- http://www.truetime.com -- products for about eight
years and have been completely satisfied; they make excellent stuff.
Right now, we're using one of their NTS-200 time server appliances:
Hook one end into an antenna cable and the other into your local
ethernet and you've got a Stratum-1 GPS-referenced NTP server, out
of the box.  If you need better, specs of their GPS-TFM device:

    UTC(USNO): <40ns rms (150 ns peak)
    3x10^-12 when tracking satellites
  Stability (tracking satellites):
    1x10^-9 at 1 second
    3x10^-10 at 100 seconds
    1x10^-12 at one day
  Stability (not tracking satellites):

GPS itself doesn't do all that much better than this.
TrueTime also sells PCI cards with 1uS accuracy, although
with Windows drivers only. :-( But there are many such
devices out there on the market, and many if not most
support Linux these days. Some simply look like UARTS.

> Still, even if one has to buy a really good digital input card per host,
> buy some signal splitters and logic, and write some drivers, one could
> probably get to 1 microsecond or better by averaging over a long time
> base and successively refining the clock setting (the central limit
> theorem is your friend) at a cost of maybe $1000 plus a card per machine
> to pipe the signal in on.  Maybe even less.

Hmmm... If you're trying to get ns-level resolution out of a
1pps signal, I'd think that you'd want an analog input that
can oversample the hell out of the square wave. But perhaps that's
what you mean by a "good digital input card"... ;-)

Also, unless your computer room has an open roof, the antenna
distribution is likely to become an issue. The highest-factor
splitter I've ever seen is eight-way


and if you're after this level of synchronization, you start to
have to worry about latency through the splitter electronics.

> Much less than a microsecond I think would require serious money,
> though.  A real atomic clock sync'd with NIST at NIST and carried to
> your machine and interfaced with it?  Then there are the kernel hacks
> one might need to start thinking about as your wall-clock time base
> resolution approaches the resolution of your CPU clock.  Computer
> latency already kills most internal timings at >microsecond resolution.

Nah, I don't think that the serious money starts until you want
better than 50ns deviation from UTC. GPS itself only can be relied
on to about 20ns or so:


and even the USNO master clock deviates from the official UTC
standard by as much as a few ns:


> I keep waiting for somebody to realize that with adequate spactial
> localization and time and phase synchronization (which can be provided
> now by GPS) that the array of cellular phone antennae that have sprouted
> all over the landscape like mushrooms have the potential to form the
> world's largest and most precise radio telescope.  If a computer with a
> really accurate time base were set up at each precisely located antenna,

Duh, how 'bout the cell phone itself? Soon it ought to be straightforward
to put a few more MFLOPS into those things for an extra 75 cents or so.

> and all the computers were linked into a giant integrated beowulf by
> means of the handy optical fiber data line that I imagine that goes into
> each of these sites, and each computer were set so that it could write
> the output of its precisely tuned radio signal input onto local disk

Forget local disk. Think flash memory.

> WITH AN ACCOMPANYING TIME BASE (to provide the essential phase shifts)
> which it could then send to a "master" node for image processing and
> interference/reconstruction, one could take >>bright<< radiographs of
> the sky with >>very<< fine angular resolution, diffraction limited by
> the fraction of the continent exposed to the source (effectively 5000 km
> or thereabouts for targets overhead) and the dispersion of the turbulent
> atmosphere (with the latter highly averaged).

OK, but *YOU* get to run the filesystem backups, understand?

> The scientific possibilities of such an array are mind-boggling.  For
> one thing, it might actually make "Star Wars" a technologically feasible
> proposal instead of a joke, as it would permit precise localization of
> incoming radio sources or reflectors.  Inverted into phase locked
> transmitters, the array might be able to focus significant radiated
> power in a highly directional manner.  It would also let us look far,
> far away and long, long ago, which physicists and cosmologists and
> astronmers always like doing.

Somehow this reminds me of the Human Genome project. So does the first
person to figure out for sure how the Universe was formed get to patent
the process? Do we then have to pay him/her royalties? As you say,
the possibilities are mind-boggling...

> I'm not sure microsecond time resolution would suffice -- nanosecond
> would be much better, opening up all of the AM bands and at least some

Given that the USNO can't track UTC by closer than a dozen ns or
so by averaging 50 cesium frequency standards and a dozen hydrogen
master clocks, this would be an interesting challenge indeed.

> of the FM bands -- but it would be fun to build with large supplies of
> other people's money.  There's a wild idea for a "beowulf" -- an
> integrated computer/radiotelescope/death ray, built out of "commodity"
> components like cellular towers and computers and very accurate
> clocks...

Methinks you've been reading Slashdot a bit too much :-)

> </bs mode=on,option=ignore>

Phew, glad that's over :-)


http://www.ntp.org/ (AKA http://www.eecis.udel.edu/~ntp/)


More information about the Beowulf mailing list