[Beowulf] Often favorable to hire HPC specialists over more hardware

[Robin Whittle]

Parts of the "Roadrunner shutdown" thread developed into a discussion
about the benefits of hiring specialized HPC programmers as an
alternative to spending more money on hardware.

For the benefit of folks searching the archives, here is one of the
messages in that sub-thread, from Brian Dobbins, which contains many
points of interest:

  http://www.beowulf.org/pipermail/beowulf/2013-April/031271.html

including:

  Why it may be worth hiring a good developer if there are a moderate
  to high number of nodes in the system already - with rough dollar
  cost estimates.

  A number of programming- and compilation- level things which can
  easily be fixed by a programmer with the right expertise, which
  will greatly improve performance, but which might not be known by
  programmers who are primarily interested in science.  Also, how
  common they are.  (I wonder: if this is true, then perhaps half or
  more of HPC cluster time is being wasted on this stuff, in
  environments where many types of jobs are being run.  I figure
  the people who devote clusters to one or a few programs probably
  invest in writing and tweaking those programs well.)

  Having a skilled programmer to ensure to that the results are
  correct, not just that there are results at all, or results
  faster than before.

  "So that's my lengthy two cents in defense of why it's /very often/
   favorable to hire HPC specialists over more hardware . . ."

I know many people tend to like short chatty messages on mailing lists,
but I like ones like this, for which Brian's apology: "(PPS.  Sorry for
the length!)" absolutely does not apply.

 - Robin