synchronizing sound cards in a cluster

[Jim Lux]

At 02:32 PM 3/13/2003 -0500, Robert G. Brown wrote:
>On Thu, 13 Mar 2003, Jim Lux wrote:
>
> > Anybody have any good ideas on how to synchronize the sampling from
> > multiple sound cards in a cluster using Ethernet as the interconnect. The
> > application would grab data from the sound card (notionally at 100
> > ksamples/second total, for two channels) and do a ton of signal
> > processing.  At some point in the processing, the streams of data need to
> > be shared between processors (i.e. to do beamforming), and so, needs to be
> > time registered.
> > The bandwidth isn't a real challenge here (with, say, 16 processors, 
> that's
> > only about 32 Mbps total), nor is latency, but synchronization is.
> >
> > One can fairly easily synchronize to a millisecond over Ethernet, but this
> > application needs sync to, at worst, 1 sample time (20 microseconds)
> > although order of a microsecond would be nice.
>
>a) Check out the documentation on http://www.ntp.org/documentation.html.
> From what it says, you can synchronize at roughly the level of network
>latency with ntp alone, so you can (I would expect) get an otherwise
>quiet LAN sync'd to a millisecond or even less.  NTP does correction
>over a long time and damps to a common clock, so you MIGHT get down
>below the 1 ms mark over time.  I doubt that ntp alone would make 10
>usec.


That was where I got my "easy to syc to a millisecond".. I've actually done it


>b) Do you get to spend money?  Can you purchase each node its own GPS
>clock?  The ntp docs suggest that if you have any sort of reference
>clock (atomic, GPS, time pulse) your resolution is limited only by the
>reference (and, probably things like gettimeofday, which are no better
>than 2 usec as it is and can easily be worse).


Better than that, one doesn't need absolute sync for this application, so a 
common signal distributed to all nodes might work.  A lot of the NTP 
"clients" can take a "tick" on one of the serial port lines (like CD or 
DTR), but I was wondering if anyone had actually done this on a cluster.


>c) If you don't get to spend money then you could TRY to use the onboard
>tsc instead of coarsely adjusting the system clock per se.  I'm using it
>as a timer in my benchmark code and can give you a wrapped assembler
>fragment for reading it.  This clock is accurate to an inverse clock
>(typically sub-nanosecond these days) BUT by the time you add the
>overhead of reading it you diminish to perhaps 40-60 nanoseconds.
>Still, in principle you have access to a clock with sub-usec resolution
>(you can even measure and correct on average for the time required for
>the wrapped call).

In fact, this is sort of what NTP and all the related things do..


The real question is whether one can synchronize the sampling of the sound 
card.  Synchronizing the processor to better than a microsecond is fairly 
straightforward (one could always use some interrupt line, for instance), 
but the interaction with a sound card is fairly complex.