Apps & Design

[Kragen Sitaker]

Alan Ward writes, quoting Gregory Warnes:
> > Careful.  You are equating "the problem consists of independent pieces"
> > with parallelizable. 
> 
> No. I'm saying that parallelizable implies we have to be able to split it
> into independent pieces, but not the other way round.

If, by "independent", you mean that there is no serial dependency ---
where the outputs of one piece are the inputs of the other --- then you
are right.  It sounds like you mean something different, though.  Can
you explain what you mean?

> Since the translation
> problem cannot be split into "large" independent pieces, it follows that
> the Beowulf approach in not efficient in this case. 

>From looking at SYSTRAN's output, it appears that the translation of
one section of a document has little effect on the translation of other
sections.

However, as Gregory Warnes pointed out, translating multiple documents
is trivially parallel.

> Maybe at lower level (e.g. pattern searching, looking up words in a table)
> the problem is parallelizable. But then we need access from all processing
> nodes to all the data, and the bandwidh necessary goes up. This would be
> more for a multiprocessor machine; say 1024 CPUs sharing RAM through
> some sort of multidimensional lattice structure. Not off-the-shelf
> components.

1024 CPUs sharing RAM is a bad idea, unless possibly they're Cray MTA
CPUs.

(What kinds of parallel programs have you written, by the way?)

> > Stating that natural language processing is inherently non-parallel
> > ignores the fact that the only system known to correctly process natural
> > language (the brain) is made up of a lot of independent processing units!
> 
> But do these units work in parallel? My take is they work more as a mesh.
> Then again, I'm not a biologist and can't prove this impression.

In the sense that we use the term in parallel computing circles, they
work in parallel.  This simply means that more than one of them is
working at any given time.  Each individual neuron can do nothing
interesting on a time-scale of less than ten milliseconds or so.

-- 
<kragen at pobox.com>       Kragen Sitaker     <http://www.pobox.com/~kragen/>
The Internet stock bubble didn't burst on 1999-11-08.  Hurrah!
<URL:http://www.pobox.com/~kragen/bubble.html>
The power didn't go out on 2000-01-01 either.  :)