[Beowulf] Stroustrup regarding multicore

[Perry E. Metzger]

"Lux, James P" <james.p.lux at jpl.nasa.gov> writes:
>>It is not uncommon for me to get someone asking me a question of the
>>form "I need to do X, how can I do it", and while explaining to them
>>how to do X, further questioning slowly reveals that they really
>>should be doing Y. It just isn't possible for someone who doesn't
>>program as well as a decent programmer to put together good
>>requirements because they don't understand well enough what is
>>possible.
>
> But that merely points up the need to develop requirements gathering
> or generating skills...

It points up the fact that the requirements gathering skill is
equivalent to the being-able-to-do skill, unfortunately.

I've also done a fair bit of management over the years. The best
technical organizations are managed by people who understand what all
their reports do, and who could in theory do their jobs. It is almost
impossible for non-programmers to manage programmers well, for
instance.

I'm not saying that this is always what happens. It is just that the
usual, non-ideal case, is amazingly inefficient. I've seen people
spend months of their lives working on code that turned out to do the
wrong thing because the domain expert didn't understand enough to know
what they actually wanted.

"If only you had told me that the database was small enough to fit in
a large main memory, we could have spent a fraction of the resources
buying a machine with 16G of memory instead of trying desperately to
optimize." "If only you had told me that the instrument was
programmable, we could have had it spit out the time stamps along with
the data points and then we wouldn't have had to waste four months
trying to fix the latencies to the machine adding timestamps." "If
only you had explained the problem better, I would have partitioned
the sparse 3D space with octrees and we could have saved 90% of RAM,
90% of the intersection calculations, and the problem would have fit
in core to begin with and would have run 2000 times faster." "If only
you had told me the actual problem instead of handing me an algorithm
to implement, I could have used a perfect hash for the filtering round
and then we wouldn't have had to use 10 computers for a week, it
would have run on my laptop in an hour."

Having non-computer scientists specify what computer scientists should
do is somewhat like having a layman tell a surgeon what to do to a
patient without the surgeon being let in on what the underlying
complaint was. The surgeon might expertly remove the person's right
kidney, but if a doctor had been giving the direction, the fact that
the pain was actually appendicitis and not a cancerous kidney might
have been noticed. To fix this, you can teach the layman quite a
lot -- at which point he's a doctor.

By the way, this isn't special to computer science. Imagine if laymen
were directing the physics research . Progress in physics might be
rather slowed down, don't you think.

>>Over the years, I've worked in several very specialized fields. The
>>people who wrote the best software were both actual computer
>>scientists and actual domain experts. I admit that such people are
>>rare, and you usually can't get them, but that is the *ideal* case.
>
> Sure, it's the ideal, but generally unachievable, case. So, given
> that we have finite resources, and all domain experts aren't expert
> coders, what's the next best thing...

I don't know what the next best thing is, sadly.

I just spent several years of my life becoming a domain expert (I was
already a computer scientist) because it seemed impossible to get the
job done by working with a non-programming domain expert, so I'm
clearly the wrong person to ask.

I admit that most people aren't going to take three or four extra
years of their lives to learn programming if they're already a
scientist or (as I did) learn science if they're already a
programmer. Clearly, as you note, most people have to limp along doing
things in the non-ideal case of programmer-who-doesn't-grok-domain +
domain-expert-who-doesn't-grok-software.

I don't know how to make that work well, though, so I didn't try. That
doesn't mean that someone out there doesn't know how to make it work
well -- but that someone isn't me.

I think what most organizations end up doing is just "living with it"
-- i.e. things don't work well but they muddle through. Every once in
a while a small disaster will happen (often unnoticed, with people
simply deal with spending $1E5 dollars on a problem instead of $1E3
dollars), and they don't know better and just muddle along. A lot of
life seems to be about muddling along anyway, so this is hardly
unusual, though it is unfortunate.

Perry
-- 
Perry E. Metzger		perry at piermont.com