[Beowulf] A start in Parallel Programming?

[Jim Lux]

At 11:55 AM 3/13/2007, Robert G. Brown wrote:
>On Tue, 13 Mar 2007, Peter St. John wrote:
>  instead of {} is just dumb.  Almost
>as dumb as
>
>       DO 100 I=1,N
>         ....
>100   CONTINUE

Which syntax compiler writers hate, because you can't use a parser 
that always moves forward.  (But compiler writers love FORTH)



>Pascal was, and remains, the German of compilers.  All sentences must at
>the end a verb have.  Declarations and definitions of all entities must
>occur in strict order. Differences in passing by reference vs value,
>especially for functions a la your example below.  Pascal made me
>shudder (back when I enthusiastically tried it).  I could almost hear
>the jackbooted heels tromping up to me door when it delivered its
>compiler warnings about some silly little infraction I'd made in syntax.
>
>All this is why CPS departments loved it.  It FORCED you, by jiminy, to
>learn structured programming -- it wouldn't compile unless you'd
>structured your code according to its inflexible and precise rules.

And it made an excellent language for that purpose.  Sort of like 
learning to drive on the correct side of the road on a freeway, 
before being turned loose on a road racing or rally course.

And, if you think Pascal was rigorous, Ada was Pascal on steroids.. 
all to try and meet the admirable goal of creating software that, if 
it compiled, was relatively bug free.


>The more important differences are associated with efficiency and
>convenience and how well the syntax matches the problem.



>Fortran is good because it tells YOU how matrices are going to look and
>what their indices will run over, and by jolly darn it you'd better
>learn to live with it.  That makes compiler-writers very happy, as they
>can optimize the hell out of matrix operations because they KNOW that
>the matrix is rectangular and starts at 1 and has a fixed layout in
>memory of its rows and its columns.  It also has a binary exponentiation
>operator, which annoys C purists because it is properly a library
>function and not an operation (a trancendental call no less) but is
>convenient as all hell when writing code and I miss it.  I do NOT miss
>doing anything at all to characters in fortran (noting well, Jeff, that
>I last wrote in Fortran -- willingly -- back when F77 was embarrassingly
>new).

And, of course, BASIC also has the same array mechanism as FORTRAN 
(but adds dynamic allocation, but no pointers)


>Except that it gives up all of the much greater efficiency of Fortran
>for numerical code.  Has C >>ever<< beaten fortran on non-trivial
>numerical code?  I doubt it. I seem to recall Greg remarking on this in
>flame wa.. I mean "discussions" past;-).  IIRC his point from that time,
>C's very flexibility makes it much more difficult to optimize.  When I
>create a typedef for a struct with three doubles and two ints as
>contents, malloc a block of memory for a vector of the structs, and then
>try to do linear algebra using the struct's second (y) double component,
>the compiler simply CANNOT know that the y's are in a simple vector with
>char stride 3*8 + 2*4 -- if that is indeed correct for the hardware in
>question.  Fortran arm-twists one to allocate the y vector as a
>standalone vector of stride 8 (or 1, double precision, using completely
>standard and built in offset arithmetic) and even starts the offset
>predictably, where I can make the offset into the C struct anything I
>like, or can drop whole vectors of variable length at EVERY point inside
>the struct by making the struct hold pointers and descriptors for the
>vectors like length and type).  Compilers can even align the variable
>favorably in memory where that matters -- what can they do with my
>struct and its mallocs (or C++'s even more hidden equivalents).



Precisely.. the FORTRAN compiler can make use of real hot rod 
hardware (think of vector processors) without you, the coder, having 
to be much aware of it.



>In some cases the C (or C++) can be much, much easier to write and think
>about because it lets you craft arbitrary objects that match the problem
>instead of taking a much smallser set of built in objects that can be
>handled efficiently and them matching them, however crudely, to the
>problem.  C++ extends the latter still further, at still greater cost in
>efficiency (although in many cases it can be programmed to be relatively
>efficient, as can the C for that matter).


The usual argument is that so what if the computational efficiency is 
lower, Moore's law means that raw computing horsepower gets cheaper 
so fast, it's better to be able to code "something" that works 
quickly, saving money on expensive software weenies... (You can buy a 
lot of computers for the $250K/yr a body costs you)

And this works if you are only going to run that task once.  Once you 
start doing it over, and over, and over again, (or the task gets 
*very* large, computationally), then more efficient languages start 
to be more important.

Compare the Matlab/FORTRAN dichotomy...  Matlab: interactive, easy to 
develop numerical algorithms, not a ball of fire speed wise, but 
typically you spend more thinking time than cpu time.  FORTRAN: 
batchey, constrained syntax and programming model, real fast on 
numerical problems.


>And then, as you note, there are really different languages -- APL,
>LISP, TCL, mathematica, python, perl -- languages where one "can" often
>program anything you like, but where the language itself is very, very
>far from the machine code produced and very, very difficult to optimize.

But these are all really "scripting" languages.. here's a sequence of 
things to do...where the things tend not to be all the same

>Basically, if you use one of them for a problem, you are acknowledging
>that you don't much care about the low level efficiency of the language,
>but that instead you REALLY care about how hard the program is to write,
>how tightly it fits the problem.



>I say fundamental because I personally like to be able to "see" through
>the compiler to the underlying assembler.  With C that is quite simple
>-- I have a fairly good idea of just how each loop is implemented, how
>memory is laid out (in considerable detail), how loops roll or unroll.

Even easier in FORTRAN, with memory statically allocated in nice 
rectangular (or cubical) blocks.

>I can exert even more control than I usually do by inlining assembler or
>using pointers even more heavily than I do.


Pointers? Anathema! I'll bet you even do things like topological 
sorts and other stuff out of Knuth, instead of nice clean matrix 
math.  If you don't know how big your arrays are and how many you 
need before you start, then you don't understand your problem well 
enough.  Dynamic allocation is a sign of weak will, or worse.

We are piling the wood for the auto da fe right now...


>[As he staggers to the refrigerator, takes out a strange looking shiny
>cylinder and presses it into a specially designed hopper on his suit.  A
>bead or two of sweat is visible on his brow as he takes a hasty swig of
>cooling liquid through a special fitting in the front of his mask.
>Overhead, drones fly relentlessly overhead, looking for some sort of
>signature, a ventilation pipe perhaps, at which they can launch their
>diabolical payload.  Cripes!  He jumps up and pumps the handle to try to
>retract the ventilation pipe, sour foam spewing from the fitting and
>drooling off of his face and down towards his toes.  Was he in time?]

James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875