[Beowulf] Re: vectors vs. loops

[Ben Mayer]

The point of the study was teaching them parallel programming with MPI
and UPC to see what the (dis)liked/(mis)understood to gain a better
understanding of how we can help them learn. An observation was that
they wrote their code in such a way that it was difficult (or
impossible) for the compiler to vectorize the code (even with
directives).

>From the experiences of helping instruct two parallel classes, and my
own experiences learning (and observing), it is a very rare case that
a scientific code (or most any large code) will be be automatically
vectorized all of the way through, unless one has a good understanding
of computer architecture AND significant experience producing code
that the compiler can optimize for a given arch.

CVL is a library that has to be called. This means that one has to
explicitly make a function call to the lib for your code to have
vectorized segments. Function calls greatly diminish a compilers
ability to optimize. Cray's compilers (and recently gcc) will
automatically vectorize code for you, if it can. Cray's will even make
it run in parallel, across processors, to some extent. The
optimization can be greatly helped by knowing how to help the
compiler. Learning how to guide the compiler takes a good deal of time
to learn.  Unless Sony/IBM releases an awesome compiler and some great
vectorized sub-systems (a graphics render, physics engine, etc) the
Cell's (PS3) speed will be hindered greatly by code not vectorizing.
This could be a great opportunity to make a large sum of money.

NESL does not appear to have momentum. Started in 1991 and the last
paper is in 1997. They talk about running on Paragon and CM5 (I
believe that UMN had serial #1), not about Cray X1, which is what we
were using.

scandal's web site makes it look like it is a research language only.

MPI is standard, UPC has compilers from many vendors (including Cray).
MPI is installed everywhere and runs everywhere, UPC can run on
distributed memory and SSI systems, and is currently being explored
for strengths and weaknesses.

Heck Matlab might be a good choice. Everything is a matrix AND it runs
in parallel... now if only our large machine with matlab would be
stable we could give some more assignments. ;)

Ben

On 4/26/05, Andrew Piskorski <atp at piskorski.com> wrote:
> On Tue, Apr 26, 2005 at 10:01:25AM -0500, Ben Mayer wrote:
> 
> > I actually just did a small study of how well students in a parallel
> > computing class write parallel codes on X1 with MPI and UPC. One of
> > the things that stood out is that they tended to do odd things in
> > their loops that inhibit code from vectorizing.
> 
> So, why were these students writing loops in the first place?  If the
> goal is to generate vectorized code, wouldn't it make more sense to
> use a language or library which directly supports vector commands?
> 
> E.g., although they're used for serial not parallel programming, S and
> R are vector oriented in a pleasantly convenient way.  There do exist
> languages and libraries specifically intended for vector programming,
> like CVL or NESL, right, so, are they not useful?
> 
>  http://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/scandal/public/papers/cvl.html
>  http://www-2.cs.cmu.edu/~scandal/
> 
> --
> Andrew Piskorski <atp at piskorski.com>
> http://www.piskorski.com/
>