[Beowulf] transcode Similar Video Processing on Beowulf?

[rf at q-leap.de]

>>>>> "Roland" == Mark Hahn <hahn at mcmaster.ca> writes:

    >> successfully) it is now ready to securely manage stuff in the
    >> cloud. Our secure remote execution engine is also based on
    >> ZMQ/Curve.

    Roland> interesting, though queues seem like a potentially odd
    Roland> choice.  don't you normally want RPC-like behavior, rather
    Roland> than just queueing?  (I use SSH to provide my secure
    Roland> cloud-enabled RPC mechanism - obviously not tuned for
    Roland> low-latency, but OTOH it's ubiquitous and solid...)

In our use case there are no queues involved (they can explicitly be
created by using a ZMQ device if needed). For the remote execution
engine, we use the simple REQ/REPLY model to
communicate between control (head) node and worker nodes, so in essence
something like a RPC. ZMQ provides reliable messaging and the security
(encryption/authentification) layer for us.

We have added additional functionality so that the control node
remembers whether commands have already successfully been executed on
the list of nodes selected. If not it will be executed once it's
reachable again, hence guaranteeing a homogeneous node state. But this
is outside of the scope of ZMQ.

To achieve the transparent multi-user support for our client GUI
(i.e. ADMIN A sees changes done by ADMIN B in real-time) in addition to
a REQ/REPLY channel, we have a separate SUBSCRIBE channel where state
updates are being sent to the GUI.

Roland