[Beowulf] Cluster Applications in Submarine Environment

[Jim Lux]

At 10:37 AM 12/4/2005, Mark Hahn wrote:
> > contact me directly.  I would particularly like to hear from anyone doing
> > parallel DSP, acoustic signal processing, or cryptography.  I'd also be
> > interested in comments regarding low power per cubic foot packaging since I
>
>as I'm sure you're aware, low-power commodity processors are also
>lower-speed.  that's certainly the case with Orion stuff, PC104 clusters,
>etc.
>
>have you considered using the new generation of relatively easy-to-program
>FPGA's instead?


Having gone through this particular tradeoff numerous times, here's a 
couple factors:

1) For a given computation, nonprogrammable hardware (e.g. the ALU in a DSP 
processor or even in a microcomputer) will be faster and/or lower powered 
than programmable hardware (e.g. FPGA).  The tradeoff is that the 
generalpurpose CPU widget might have stuff you don't need (and that you 
could leave out of the FPGA).  Do not believe, without seriously checking, 
claims of "you can put N PowerPC cores on a model Y FPGA" and think that 
this means its the same as N PowerPCs.. Lots of little details like memory 
and peripheral management, caches, lookahead pipelines, etc.

2) FPGAs are by no means easy to program (if you want to achieve the 
potential performance/power advantages). The tool sets are more expensive 
(except for bottom of the line FPGAs), most particularly if you need 
diagnostics and simulators (there are "free-ish" compilers that will turn 
Verilog and VHDL into a bit stream for the FPGA.. but you need a lot more 
than that to do serious work.).. This is getting gradually fixed by the way..

3) There are orders of magnitude more people around who know how to write 
good software than how to generate good code for a FPGA.  This translates 
to lower costs, less schedule risk, etc.

4) The FPGAs flexibility is very much a two edged sword.  You can generate 
highly efficient designs with little waste.  You can generate incredibly 
complex and untestable designs.  The level of diagnostic tools to "peer 
into the insides" of the FPGA logic is limited in practical terms, so you 
might be reduced to combining little testable pieces into an inefficient 
whole.  Think of designing firmware for FPGAs as akin to designing digital 
logic with MSI TTL parts.  A million gate FPGA might correspond to a 
thousand or so fairly complex MSI devices (UARTs, MCUs,e tc)  Do you feel 
comfortable designing and debugging a logic design that occupies a dozen or 
so E-sized sheets?

5) The cycle time (revise code, recompile, load, test, look at results) to 
test something is slower for FPGAs than for straight software. A moderately 
complex design (say, something that fits in a several hundred thousand or 
million gate FPGA) might take hours to resynthesize and compile.  If you're 
using SRAM based FPGAs (Xilinx) the load time isn't all that big (although 
streaming in a few million bits over the JTAG port might take a while, 
depending on your support hardware.)  If you're using fuse/anti-fuse 
programmable FPGAs (e.g. Actel), it could take longer (and, since they're 
not reprogrammable, it's pricey to have a lot of revs)

6) The purchase cost per gate of an FPGA is much higher than that of a 
massproduced CPU (DSP or general purpose), so to come out ahead the CPU 
approach has to be a LOT less efficient than the FPGA, or, the FPGA has to 
provide some unique capability that just isn't available on the CPU (e.g. 
hardware implementation of some algorithm that doesn't lend itself to 
efficient software/CPU processing... DES used to be in this 
category).  FWIW, this turns around when you get to ASICs.. so you may 
start with a FPGA design that eventually turns into an ASIC if the volumes 
justify.

7) FPGAs are very nice for very high integration of disparate functions.. 
you want to combine a IEEE-1394/Firewire interface, a high speed A/D 
control interface, and a MPEG-2 encoder and some other stuff into a single 
package.  Another example might be something like integrating a high level 
controller with multiple low level motor control loops.

8) If you have hardware that needs to be controlled/interfaced, the fact 
that FPGAs can provide a lot of the glue at small additional cost is very 
attractive.  Think in terms of a radar signal processor.. you need to talk 
to the A/Ds and apply a well defined repetitive processing to the data, and 
furthermore, one that has already seen a lot of development in terms of 
hardware implementation at the gate level, so you can leverage existing 
designs.  There's a lot of essentially off the shelf ways to do useful 
stuff in FPGAs: oscillators, filters, interpolators, FFTs, etc.



>  the real beauty of Beowulf-like hardware is its flexibility
>(as well as commodity pricing, of course).  but your application seems much
>more of a production thing, where the greater compute-density of FPGA's
>would be appropriate.  costs would depend on scale, I think.
>
>regards, mark hahn.
>

James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875