[Beowulf] A start in Parallel Programming?

[Robert G. Brown]

On Tue, 13 Mar 2007, Jim Lux wrote:

>> Pascal was, and remains, the German of compilers.  All sentences must at
>> the end a verb have.  Declarations and definitions of all entities must
>> occur in strict order. Differences in passing by reference vs value,
>> especially for functions a la your example below.  Pascal made me
>> shudder (back when I enthusiastically tried it).  I could almost hear
>> the jackbooted heels tromping up to me door when it delivered its
>> compiler warnings about some silly little infraction I'd made in syntax.
>> 
>> All this is why CPS departments loved it.  It FORCED you, by jiminy, to
>> learn structured programming -- it wouldn't compile unless you'd
>> structured your code according to its inflexible and precise rules.
>
> And it made an excellent language for that purpose.  Sort of like learning to 
> drive on the correct side of the road on a freeway, before being turned loose 
> on a road racing or rally course.

If they'd only taught the courses using individuals wearing boots of
shiny shiny leather and carrying a riding crop... it might have been
popular.  Or to abuse your metaphor instead, sort of like being taught
to drive on the correct side of the road by hitting your car in the side
with a bazooka if it ever strayed across the centerline.

I personally would just rather wreck the damn car on my own a few times
and have to fix it, but to each his own...:-)

> And, if you think Pascal was rigorous, Ada was Pascal on steroids.. all to 
> try and meet the admirable goal of creating software that, if it compiled, 
> was relatively bug free.

<shudder>

And really old people also remember Our Old Friend, PL/C, which never
met a line of code it didn't like.

I actually watched somebody pull a bunch of cards (including mistyped
cards) out of the trash and compile them just to show that it could be
done.  It would cheerfully add ; to the end of lines, try to puzzle out
what the hell command was somewhat "like" the strings on the card, and
dutifully produce, well, an executable object, if properly intructed to
do so.  It wouldn't run, of course, but damn it would compile.

I really liked PL/1.  Again a relaxed interpolation of C and fortran, in
a lot of ways.  Fortran IV (which I learned after PL/1) really sucked in
comparison.  Noting that this was pre-1977 (wince).

> And, of course, BASIC also has the same array mechanism as FORTRAN (but adds 
> dynamic allocation, but no pointers)

I actually was at one time moderately fond of BASIC.  It was a kind of
"relaxed fortran" syntactically, and in PC form could actually manage
strings and so on fairly nicely.  Then I learned C and realized that one
simply couldn't do any better.  Unless one added a binary exponentiation
operator and inlined perl-like regular expression parsing, of course,
which might mess it up for systems code but damn would make it fun to
program.

> Precisely.. the FORTRAN compiler can make use of real hot rod hardware (think 
> of vector processors) without you, the coder, having to be much aware of it.

Provided, of course, you're willing to work with the much smaller
palette of data objects that it can actually optimize in this way.
Which at times means that you're going to waste a hell of a lot of space
or work with inside-out data objects OR have to do all sorts of evil
with reindexing schemes, which -- oh dear -- means you have to be aware
of it after all.

It is sometimes very instructive to port a bunch of Fortran to C (not
using openly diabolical tools like f2c, but the hard way).  Parts of the
port are trivial as parts of the syntax are either identical or a
trivial map from a to b.  The interesting places occur in nontrivial
code where one isn't just looping over vector objects (which I'd argue
is overwhelmingly Fortran's Forte).  One frequently ends up going "huh?"
and completely rewriting some data structures that one realizes are
horribly INefficient in Fortran, simplifying the logical flow, and
sometimes shortening the hell out of the code.  Of course one can often
then go back and rewrite the Fortran to simplify it as well (and it may
be harder to do this as of F90 and above, as they've simply swallowed
enough C-isms that you can't do this as much).  I've spent quite a bit
of time doing this -- e.g. porting diehard into C, and it is, in the
end, maddening.  Makes one want to commit acts of violence and drink
heavily.

>> In some cases the C (or C++) can be much, much easier to write and think
>> about because it lets you craft arbitrary objects that match the problem
>> instead of taking a much smallser set of built in objects that can be
>> handled efficiently and them matching them, however crudely, to the
>> problem.  C++ extends the latter still further, at still greater cost in
>> efficiency (although in many cases it can be programmed to be relatively
>> efficient, as can the C for that matter).
>
>
> The usual argument is that so what if the computational efficiency is lower, 
> Moore's law means that raw computing horsepower gets cheaper so fast, it's 
> better to be able to code "something" that works quickly, saving money on 
> expensive software weenies... (You can buy a lot of computers for the 
> $250K/yr a body costs you)
>
> And this works if you are only going to run that task once.  Once you start 
> doing it over, and over, and over again, (or the task gets *very* large, 
> computationally), then more efficient languages start to be more important.
>
> Compare the Matlab/FORTRAN dichotomy...  Matlab: interactive, easy to develop 
> numerical algorithms, not a ball of fire speed wise, but typically you spend 
> more thinking time than cpu time.  FORTRAN: batchey, constrained syntax and 
> programming model, real fast on numerical problems.

I have no problems with any of this, and there is certainly some truth
to it -- in certain arenas.  OTOH, the end stage of this attitude is, to
be frank, Vista.  A program that is so huge, kludgy, and inefficient
that it runs like a pig on the fastest, biggest hardware in existence,
and it is >>just the bloody operating system<<, the part that has to
basically manage a core execution loop, a scheduler, some devices, and
be able to load and run software.  God help anybody that actually needs
to run an application when the OS itself is designed by people that
don't give a rat's ass for elegance and efficiency.

Even linux is starting to suffer from a certain amount of bloat.  The
primary difference is that linux is still designed in layers, and one
CAN run a considerably stripped linux and CHOOSE to layer on the bloat
that you need.

I personally think that there is a relatively clean line out there
somewhere in the sand -- for quickies and one-offs, use a scripting
language (and I don't even care which one).  If you're going to use a
compiler, though -- or write something even vaguely commercial -- then
by golly think hard about elegance, efficiency, and stability.  And yes,
maintainability, but that truly is a function much more of coder
competence in any language than it is of language.  You can write
beautiful elegant code in an ugly language, and ugly code in a beautiful
one.

>> And then, as you note, there are really different languages -- APL,
>> LISP, TCL, mathematica, python, perl -- languages where one "can" often
>> program anything you like, but where the language itself is very, very
>> far from the machine code produced and very, very difficult to optimize.
>
> But these are all really "scripting" languages.. here's a sequence of things 
> to do...where the things tend not to be all the same

Indeed.  One of the primary differences being addressed is that creating
a relocateable program that links to dynamic libraries is a whole
different beast relative to taking a character string, parsing it, and
taking actions that amount to calling prebuilt subroutines on internally
managed data structures.  One is, or can be, efficient, elegant,
maintainable, stable.  The other cannot be as efficient, may be as
elegant or far MORE elegant, and is maintainable and stable to the
extent that the language itself is.  Scripting languages tend to be a
bit shakier than compilers in this latter regard, although of course by
now there are some venerable ones that are quite stable and bug free.

>> I say fundamental because I personally like to be able to "see" through
>> the compiler to the underlying assembler.  With C that is quite simple
>> -- I have a fairly good idea of just how each loop is implemented, how
>> memory is laid out (in considerable detail), how loops roll or unroll.
>
> Even easier in FORTRAN, with memory statically allocated in nice rectangular 
> (or cubical) blocks.
>
>> I can exert even more control than I usually do by inlining assembler or
>> using pointers even more heavily than I do.
>
> Pointers? Anathema! I'll bet you even do things like topological sorts and 
> other stuff out of Knuth, instead of nice clean matrix math.  If you don't 
> know how big your arrays are and how many you need before you start, then you 
> don't understand your problem well enough.  Dynamic allocation is a sign of 
> weak will, or worse.

<speechless gape>

>
> We are piling the wood for the auto da fe right now...

[He pulls out the shiny cylinder, now empty, and plugs in a tall bottle
of amber fluid in its place.  Sweat is now pouring down his face.  A
steady rumble of nuclear holocaust comes from overhead, and the concrete
ceiling is starting to glow a dull red.  Taking a deep pull on the mask
nozzle, his eyes begin to glaze.  Then his fingers begin to stab blindly
at the red button, the one that launches the ultimate, world-ending
reprisal.]

Right.

How could I have been so blind?  Oh please, please, I confess!  I was
tempted away from the path of goodness and righteousness by the thought
of consorting with all of those daemons... the ones in boots of shiny
leather.  I will return to the one true faith... and go buy my copy of
Vista tomorrow!

Seriously, you and Jeff can try to find a 12-step program in your
respective geographic areas.  I'm sure there are other recovering
fortran-heads.  You can share personal stories about the horror of
hollerith and the time you managed to write a whole 500 line program
with no GOTOs!

Otherwise your friends may have to hold an "intervention".  And you know
what that involves, don't you?

De-programming.

Think about it...;-)

     rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu