[Beowulf] A start in Parallel Programming?

[Joe Landman]

Jim Lux wrote:

> Precisely.. the FORTRAN compiler can make use of real hot rod hardware 
> (think of vector processors) without you, the coder, having to be much 
> aware of it.

This is quite important to making things work well.  Abstract the 
hardware.  Hide it from the user.  Make the compiler "smart" or 
something like that.  Its a SMOP ...

> 
> 
> 
>> In some cases the C (or C++) can be much, much easier to write and think
>> about because it lets you craft arbitrary objects that match the problem
>> instead of taking a much smallser set of built in objects that can be
>> handled efficiently and them matching them, however crudely, to the
>> problem.  C++ extends the latter still further, at still greater cost in
>> efficiency (although in many cases it can be programmed to be relatively
>> efficient, as can the C for that matter).
> 
> 
> The usual argument is that so what if the computational efficiency is 
> lower, Moore's law means that raw computing horsepower gets cheaper so 
> fast, it's better to be able to code "something" that works quickly, 
> saving money on expensive software weenies... (You can buy a lot of 
> computers for the $250K/yr a body costs you)

If your need for speed increases faster than Moore's law will give you 
performance for, or if you buy into the notion that there may in fact be 
some fundamental physical limits on Moore's law due to the nasty 
combination of thermodynamics (stability of small structures with 
respect to temperature, and resistance of same to defect formation), and 
  quantum mechanics (those 30-40nm structures are getting awfully close 
to the deBroglie wavelength of a 300K electron, about 7nm (hopefully RGB 
will take me to task if I messed up that calculation) ...) ...  We may 
run out of room in this expansion unless we to a technological shift.

> And this works if you are only going to run that task once.  Once you 
> start doing it over, and over, and over again, (or the task gets *very* 
> large, computationally), then more efficient languages start to be more 
> important.

Heck... more efficient hardware as well!

[...]

>> And then, as you note, there are really different languages -- APL,
>> LISP, TCL, mathematica, python, perl -- languages where one "can" often
>> program anything you like, but where the language itself is very, very
>> far from the machine code produced and very, very difficult to optimize.
> 
> But these are all really "scripting" languages.. here's a sequence of 
> things to do...where the things tend not to be all the same

Erp... APL as a scripting language ... LISP as a scripting language 
(yeah, scheme) ...

Ok, more to the point, these are all higher level (mostly non-compiled, 
though Perl does in fact compile to an internal representation, and I 
thought LISP compiled ...).  One can do some pretty serious computing 
with "mere scripting languages".  Most of these offer higher levels of 
abstraction so you don't have to deal with details that you might not 
want to.

>> Basically, if you use one of them for a problem, you are acknowledging
>> that you don't much care about the low level efficiency of the language,
>> but that instead you REALLY care about how hard the program is to write,
>> how tightly it fits the problem.

Yup, for the most part.  Though some supposedly high level abstract 
languages make this far harder and more verbose than it should be.

[...]


>> I can exert even more control than I usually do by inlining assembler or
>> using pointers even more heavily than I do.
> 
> 
> Pointers? Anathema! I'll bet you even do things like topological sorts 
> and other stuff out of Knuth, instead of nice clean matrix math.  If you 
> don't know how big your arrays are and how many you need before you 
> start, then you don't understand your problem well enough.  Dynamic 
> allocation is a sign of weak will, or worse.

Egad man... have you not heard of the abuses of the fortran equivalence 
statement?

> We are piling the wood for the auto da fe right now...

Ah... humor ....



-- 

Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 734 786 8452 or +1 866 888 3112
cell : +1 734 612 4615