[Beowulf] IEEE 1588 (PTP) - a better cluster clock?
James.P.Lux at jpl.nasa.gov
Fri Jul 20 16:41:01 PDT 2007
At 01:07 PM 7/20/2007, Patrick Geoffray wrote:
>Patrick Ohly wrote:
>>That's all for now (and probably enough stuff, too ... although perhaps
>>you prefer detailed emails over bullet items on a PowerPoint
>>presentation). So what do you think?
>Since you are asking, here is my personal opinion: I don't think
>there is a need for a system-wide PTP service. NTP's precision is
>good enough for comparing log and other administration needs. The
>only requirement for a high precision/high accuracy global clock
>reference that I would consider legitimate is for tracing tools
>(that's where you come from). In this case, why not running NTP as
>part of the tracing environment ?
There are tons of reasons why one might want more precision than NTP
can provide, leaving aside the obvious applications for things like
controlling test equipment.
Consider if one is using distributed processing of high rate sensor
data (radar, ultrasound, seismic).. 1ms sorts of accuracies (which is
doing quite well with most NTP implementations) aren't going to hack
it for doing the time alignment. Sure, one can run discrete
synchronization interfaces, etc, but it's truly a pain.
There are also cases where one might want to perform various things
in synchronism (e.g. driving a video wall?) where the synchronization
requirement is microseconds. Say you're generating subframes of
video, and the processor speeds are all slightly different. You
effectively need some form of barrier synchronization to
synchronously update all the images.
A similar application might be real time panoramic stitching and
resampling of multiple video streams.
Sure, there are application specific approaches to these things (e.g.
genlock in the video world, various timecodes in the video, audio,
and instrumentation worlds) but they require different hardware for
different applications, provide varying levels of performance, and
aren't necessarily well integrated with other things (e.g. the IRIG
time codes derive from the days of analog instrumentation tape
recorders to get microsecond timing precision)
1588 is a nice start on doing this in a more consistent and scalable
way than current hacks like putting GPS receivers in each computer, etc.
NTP also has some problems with synchronization in relativistic
environments and ones where the delays are rapidly changing over
large magnitudes (consider trying to use NTP to synchronize a clock
orbiting the moon). While there are techniques currently used for
time synchronization over large distances in these situations, they
tend to be oriented towards determining sync for a single endpoint
(i.e. a deep space probe) and also tend to involve post
processing. They're not as appropriate for things like a
constellation of platforms (imagine a bunch of airplanes flying
around, as an example).
There's great value in any sort of IP (internet protocol, not
intellectual property) origin solution (even if it requires special
PHY layers), because IP links are becoming ubiquitous.
>You could say that it would be useful to factorize this
>functionality so that it can be shared by several tracing libraries.
>I don't think it's worth the effort, as one rarely uses two
>different tracing tool at the same time.
>I don't know enough about PTP to comment on the implementation,
>except that relying on multicast is a bad, bad idea. It does not
>scale and it's very unreliable under load (induces contention if reliable).
I'd have to go back and review the 1588 specs, but my impression is
that one can implement a PTP system in a way that is scalable to very
large sizes, otherwise, it wouldn't be much use. Indeed, such an
implementation might not coexist with other needs for the network, so
you'd be back to running parallel networks, but at least they are
more scalable than distribution amps and GPS receivers.
>Beowulf mailing list, Beowulf at beowulf.org
>To change your subscription (digest mode or unsubscribe) visit
James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
More information about the Beowulf