[Beowulf] Stroustrup regarding multicore

[Robert G. Brown]

On Tue, 26 Aug 2008, Michael H. Frese wrote:

> Vincent,
>
> I have always said that C++ is computational science's cold fusion: lots of 
> power in, but no net gain.
>
> C is not much better.  I once worked a young computational programmer for 
> almost a week to get him to prove to himself that a C source program couldn't 
> walk through a 2-d array the hard way as fast as a Fortran source program 
> unless the stepping was coded by hand. He didn't believe that a 2-d array in 
> C is syntactically a 1-d array of pointers to 1-d arrays, and the row 
> pointers must be fetched from memory!  And separate compilation of functions

Perhaps, but don't most C programmers allocate such an array as a single
vector and then repack the indices?  That way they can address it as a
vector or via the dereferenced pointers.  Does fortran do this
differently?  I mean, in C one can write a macro to do the dereferencing
arithmetic without pointers to pointers, but I thought processors and
compilers managed this very efficiently anyway.

The real point is that by using C constructs one CAN effectively code
things by hand without (usually) needing to resort to assembler inserts.
Or one can freely use inline assembler, because one has a very good idea
of what memory looks like and how C code is translating into
assembler/machine language anyway.

This isn't to say that your observation isn't apropos.  Fortran from the
beginning has been a highly structured language, more or less DESIGNED
to do linear algebra.  The compilers have therefore been tuned and
optimized for managing arrays.  C was designed to do systems
programming, where one has to manipulate bits and bytes and address
specific locations in memory.  C was also designed to make it easy and
natural to write code to process text data.  In fact, it was designed to
be a language one would use to WRITE compilers (which are, after all,
basically text processors before they are anything else).  I've actually
written a compiler-simulator in Fortran (once, long, long ago -- on
cards) and believe me, it isn't pretty:-).

Nowadays, of course, C and fortran are gradually moving towards a
convergence of some sort.  Modern fortran has absorbed most of the
really useful C coding constructs and -- perhaps -- isn't so appalling
to program to handle text data.  C has spun off C++ (and various other
things) that are "more structured", although its real advantage was and
is its LACK of structure, its tremendous flexibility, its low level
power.

My favorite description of C (due to somebody on this very list, long
ago, but I can't remember who) is that it is "a thin veneer of
upper-level language sensibilities on top of raw assembler".  I think
this is dead on the money.  As Vincent pointed out, a good C coder (one
with experience at writing assembler at some point in their career) can
"see" the likely assembler produced by each statement as they write it.
I can visualize all the loads and stores and register contents and
underlying compare statements and jump statements and stack pointer
dereferencing and data dereferencing.  The visualization probably isn't
totally accurate -- even C compilers do nonlinear stuff to optimize code
(unroll loops and so on) -- but for short, straightforward code
fragments it is probably pretty darn good.

> with variable array dimensions?  I hear echoes of Kernighan and Ritchie 
> laughing with each other "We don't need no steenking libraries with 
> execution-time array dimensioning!  We're system programmers here!  Besides, 
> if somebody needs that they'll use Fortran."

And there is nothing wrong with that.

Again on this very list, somebody (perhaps Mark Hahn?) pointed out that
C++ may be on the surface slower than C because it adds even more layers
of memory management and dereferencing in order to handle strong typing,
templating, object allocation and inheritance and all that.  A C coder
expects it to be MUCH slower because they can visualize a lot of
overhead they could skip by just malloc'ing a freeform vector of structs
and directly dereferencing instead of going through
constructor/destructor/opaque object function calls to get to its
contents.

However, C++ exploits its strong typing advantage by adding a standard
library with many, many subroutines that are carefully performance
tuned.  This permits a C++ coder to write very high level code without
much of a performance hit relative to C, at least unless a C coder uses
a lot of personal effort to write and tune their own library.  A
portable C scientific library (e.g. the GSL) de facto introduces a lot
of "objects" in order to achieve the data description uniformity
necessary to write e.g. an ODE integrator or linear algebra collection.
Otherwise I make my **matrix one way, you make it another, and one
anti-optimizes because of going through the matrix in the wrong order.

I love C, and have given Fortran up for life (after coding thousands of
lines in it, mind you) but there is no reason to expect any single
language to be universally efficient at all levels.  There are excellent
reasons NOT to expect this -- in a sense it is obvious that it would not
be so.  Assembler is infinitely powerful and very difficult to code;
matlab is relatively easy to code but very restricted and narrow in its
power.  In between one has many languages that are suitable for many
purposes, some narrow in scope but easy within scope, some broad in
scope but more difficult because breadth DOES require a trade-off
between it and speed and ease and flexibility.  C holds a place of
honor, I think, as being the most flexible and powerful upper-level
language, period -- a single notch over assembler as noted.

Fortran holds a place of honor as well within its NARROWER regime of
applicability.  Of course it is "better" than C at doing mathematical
formulas and linear algebra -- it is what it was designed to do.  But
only a madman would try to write a word processor or operating system in
it.

    rgb

>
> Sure, you may be able to use C++ to add reasonable array behavior to C.  But 
> that's like writing a yacc script to build a compiler for a language in which 
> you can write your entire program with a single executable statement.
>
>
> Mike
>
> At 11:36 AM 8/25/2008, you wrote:
>> Well Stoustrup should be the last speaking about multicores, he
>> better stick to single core.
>> 
>> Let me explain.
>> 
>> The experience learns that most C++ code from BIG companies not to
>> mention organisations is factor 5 to 50 slower
>> than an imperative implementation. Templates get used to declare
>> variables and classes are very deep, not seldom
>> 10 subclasses deep before you actually see a few lines of code doing
>> something, and you
>> never know which object gets allocated now and deallocated there.
>> 
>> Therefore for highend computations C++ is not very interesting.
>> 
>> Add to that, the C++ standard (iso) where Bjarne has contributed to a
>> lot (deep respect for that),
>> costs hundreds of dollars to buy,
>> so there is no clear hard definition easily available to programmers
>> at home to figure out the optimization
>> limitations that C++ allows. To get speed you need to know the exact
>> borders which compilers must follow.
>> 
>> int a = 5;
>> unsigned int x = 2;
>> 
>> if( a == x )
>> 
>> how C treats the above is easy. how does C++ treat it?
>> Do you know?
>> 
>> To program code for speed you need to know the EXACT limitations of a
>> language.
>> Clear definitions, not 5000 pages with what the language all "might
>> be able to do for you".
>> 
>> No one who learns at home is gonna pay to get that ISO of course and
>> the language is getting that complex and even more
>> complex, that programmers who try to learn it are busy half their
>> life learning it. After many years, just knowing how to code C++,
>> they already are too old to program a lot and feel ready to become
>> teamleader or manager. This is because C++ is the most
>> complex programming language on the planet. It has changed. I've got
>> some C++ books from start of 90s here and none of them
>> mentions things like templates.
>> 
>> The C++ from the 90s was very useful for companies, had it been
>> standardized sooner.
>> 
>> Bjarne made c++ too complex however.
>> 
>> College students who start C++ now, directly start using templates
>> and never learn how to actually DECLARE a variable anymore.
>> 
>> Object orientation is the opposite of what modern processors are good
>> at. First of all object allocation and deallocation is real
>> slow and even the best C++ programmers have problems limiting the
>> number of allocations that makes their software real slow.
>> 
>> Additionally putting together code and data, as well as things like
>> templates, makes code sizes real massive huge.
>> This where for crunching power we'll see more and more tiny
>> processors where having a lot of code is just slowing down.
>> 
>> That said, C++ has basically a number of advantages over JAVA and C#.
>> Graphical you can do the same in visual studio
>> with C++ like you can do with c++, so there is no reason to program
>> in C#. In C++ you CAN incorporate C code easily
>> as well as compiler intrinsics, even entire assembler programs (gcc).
>> So you DO have the choice to hire a programmer who
>> can speedup your code.
>> 
>> The good C++ programmers who are really good in getting code done,
>> usually have a low grade highschool,
>> no university or at most 1 year college or so, if at all and know
>> relative little about algorithms let alone optimization techniques.
>> 
>> That's your typical C++ coder. Bugfree code that's ugly slow.
>> 
>> Over the past years i've helped out dozens of PHD's who didn't know
>> how to speedup their C++ code at all. Not even where to start.
>> They know books from Stoustrup from head though and are in the
>> knowledge of all kind of details. Usually not seldom within days
>> that results in factor 2 to 3 speedup.
>> 
>> Writing yet another book with things that even most C++ scares in C+ +0x is 
>> not very interesting IMHO, and just shows how much of
>> a nerd a person can become.
>> 
>> I'd argue there is a big need for a new language that is basically
>> imperative, where there is mechanisms, but not necessity,
>> to split code and data, where you can declare anywhere new
>> temporarily variables, and which has the potential to get the same
>> speed like C code and where the language constructs are not far away
>> from C/C++, to get programmers not extra confused.
>> 
>> So instead of C++ something like Cr with the 'r' of 'realistic'.
>> Something that is really usable for companies to get fast code
>> at tiny processors meanwhile compatible with C libraries, which
>> dominate the open source world, with good reasons.
>> 
>> So just the basic minimum that you need to make out of C a language
>> that big companies can use, without losing speed.
>> 
>> Speed matters for mass software and highend.
>> 
>> C++, JAVA and C# are just too slow to take a lecture with the below
>> name serious.
>> 
>> Instead of trying to standardize the manner how to write a program in
>> C++ writing in clear statements that all kind of complex C++
>> language such as templates should get avoided at all costs, except
>> when it has a clear benefit that other simpler straightforward code
>> doesn't offer the opposite is gonna get spoken out in the speech.
>> 
>> What happens now with C++0x  is yet another nerd addition to make
>> worlds most complex language even more complex.
>> 
>> Vincent
>> 
>> On Aug 22, 2008, at 6:31 PM, Peter St. John wrote:
>> 
>>> In this interview http://www.devx.com/SpecialReports/Article/ 
>>> 38813/0/page/1 Bjarne Stroustrup  talks about  an upcoming C++ ISO
>>> standard,  C++0x  (the same nomenclature as "C89", "0x" means the
>>> specific year is undecided, it doesn't mean hex :-).
>>> 
>>> He categorizes the additions in three ways, Concurrency, Language,
>>> and Libraries; the concurrency part is about multicore support.
>>> 
>>> He writes, 'Basically, the "concurrency" features will standardize
>>> the basic layers needed to do systems programming in a multi-core
>>> world. Obviously, facilities for doing that already exist in C++
>>> implementations, but they are not standardized. I'd have liked to
>>> see library support for some high-level concurrency models, but the
>>> committee didn't have the time or consensus for that.'
>>> 
>>> Peter
>>> _______________________________________________
>>> Beowulf mailing list, Beowulf at beowulf.org
>>> To change your subscription (digest mode or unsubscribe) visit
>>> http://www.beowulf.org/mailman/listinfo/beowulf
>> 
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org
>> To change your subscription (digest mode or unsubscribe) visit 
>> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
>
> {Dad, Mike}
>
> ________
> The notation {option1, option2} means choose option1 or option2, and comes 
> from the era of command-line computer programs and their documentation.
> NYLLP = New York Local Liberal Paper aka The N__ Y___ T___s.
> Tinfoil Hats = Tin-foilers = Wackos of the Democrat base who think hatred of 
> Bush is a political principle

-- 
Robert G. Brown                            Phone(cell): 1-919-280-8443
Duke University Physics Dept, Box 90305
Durham, N.C. 27708-0305
Web: http://www.phy.duke.edu/~rgb
Book of Lilith Website: http://www.phy.duke.edu/~rgb/Lilith/Lilith.php
Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977