[Beowulf] El Reg: AMD reveals potent parallel processing breakthrough

[Lux, Jim (337C)]

>>
>>
>>
>> Top 500 is just that "top 500"..
>>
>> What fraction of total computational work done on clusters is being
>> done
>> by the top 10?   I suspect that when you get down that list a ways,
>> you
>> start seeing fairly pedestrian clusters with fairly conventional
>> nodes,
>> and that there are a LOT of them and they do a lot of work, day in
>> and day
>> out.
>>
>
>In reality industry is already crunching on GPU's for a long time.
>In industry they are more realistic than the professors on this list.
>They realize that to stay ahead of the pack you have to calculate
>more than
>the others. To do that more efficiently is always with the lowest
>level code you can
>imagine.


In the immortal style of the internet (just because the Wayback machine
will preserve it forever, not because it's inherently good):

[Citation needed]


I honestly don't have any idea how many clusters there are, or how big
they are, or whether they are standard x86 or use GPUs, etc.

My feeling, though, based on a lot of years of developing software in lots
of very different industrial environments, and benefiting from my wife's
similar experience, is that human resource and other issues often dominate
technological ones.  If you're a company that needs to fasten pieces of
wood together, and you have 100 hammer-wielders skilled at pounding nails,
and for some reason, you are forced to change to screws, you may find it
cost effective to use them to pound screws in, rather than try to hire 10
screwdriver turners and lay off the 100 hammer-wielders.

Just because something is "doable" (e.g. GPU programming) does not mean
that you can readily hire people to do it.   In my current work, we have
lots of systems that are FPGA combined with a standard Von Neumann CPU.
It is much easier (and cheaper) to hire people to program the CPU side
than the FPGA side.  Even a casual look at relative salaries on any of the
survey websites will show that FPGA coders get paid substantially more
than C coders (the fond hopes of the creators of VHDL and Verilog,
notwithstanding).  

It is much easier to test and validate the C and C++ code, as well: any
old PC serves as a test platform. For FPGA code, you need a fairly high
fidelity test platform.  I imagine the same is true of coding for GPUs.
You need one of those GPUs to test with.

No one of these issues is a deal-breaker for using exotic hardware, but
the sum total of them makes the use of anything other than plain vanilla
boxes potentially more risky and expensive. And for most (not all!)
business applications, predictability is more important than performance.




>
>Each few years new hardware means simply the hardware is more
>expensive than
>a good programmer. That was the case 30 years ago and that is the
>case today and that
>will be the case 30 years from now, simply because doing effectively
>more work at the same
>hardware means that you can see further in the future than your
>competition.
>
>That's how majority is doing it in industry for quite some time now.
>
>A bunch of them also has far more realistic energy price than the
>government.
>
>Governments have the habit to put the supercomputers nearby where
>their scientists are.


In the US, at least, *where* government spends its money is often only
partly influenced by technology considerations.  If you have a government
research lab with civil servants who don't have work to do (because some
large program has ended), then putting resources there may be a wise move,
in an overall system perspective.

Narrow optimization is not always a good strategy.

>Yet most of them had already a few gpu's inside those thousands of
>nodes, long before any box in the top500 had.
>Obviously you can calculate yourself then who has the fastest generic
>computational power. That's industry/finance obviously
>and it always was. Now finance is in a slow conversion from some
>companies doing their calculations in a stupid manner;
>some of those who did do it that way basically have been forced by
>the market to do it in more efficient manners now.

It is wretchedly expensive to change how a business does something. The
"change cost" is typically far larger than any putative "waste" by doing
things non-optimally.


>
>That's usually the opposite idea of how bunches of governments run
>their supercomputers.
>
>If in industry you outsearch your opponent by 1 second at a total
>timespan of 1 year, you already win.

Only in some very, very small fraction of industry which relies on speedy
computation.  Here on this list, we think about computation, it's what
we're all about, but the "computation" business is a tiny part of the
overall US or World GDP. For most of that business, improving computation
speed by 10% isn't going to make much difference. Look at all the people
using WinXP and IE6, and it serves them perfectly well.

At the edges, yes, faster computation helps. If you're doing FEM modeling
to design new parts to be made by your fancy CNC machining stations,
speeding up your FEM codes would be useful.  But if you consider a $100M
company making things, they probably spend 90M on the "making" and 1M on
the "designing" part of the business.  Doubling the speed/cutting the cost
in half of the 1M part of the business is 500k to the company. Improving
the speed of the 90M part by only 10% is $9M.



>
>People don't care whether your car is 100 miles an hour faster or
>0.0001 miles an hour faster. Faster is faster.

That is wrong, see example above.  Because the choice is NOT "is my car
faster" it's "do I spend time on making the car faster, or driving a
shorter distance, or... " and they all have different costs.  In
simplistic econ 101 terms, you want the partial derivative of marginal
utility with respect to investment to be equal for all components of the
business.


>
>In fact most research centers simply prefer a small cluster
>themselves instead of the total burocracy that centralized
>supercomputers give.


Sure... Particularly for research, one prefers a system which you control
entirely, rather than one which you have to share with others.
>
<snip of stuff about finance>

For all that trading and finance are in the news, there's an awful lot of
manufacturing work being done.  Something like 80-90% of all patents and
R&D are in the manufacturing sector, not finance.

>>
>>
>> Bringing up an interesting question.  How many clusters are there
>> in the
>> world? Clearly thousands.  Tens or hundreds of thousands?
>> What's the size distribution?
>>
>> 10-15 years ago, people were proud of their 16 or 32 node clusters.
>> Today, we talk about toy clusters in that size range. Limulus is,
>> what, 3
>> boards with 4 core processors?
>>