[Beowulf] Moores Law is dying

[Robert G. Brown]

On Thu, 9 Apr 2009, Peter St. John wrote:

> Yes, supersonic travel over land is restricted in the U.S.; so for example
> the SST was good for NY to London but not NY to LA.
> 
> We're no where near the limits to information per cubic centimter; but I
> don't know how to define information **processing** per cubic centimeter.
> Maybe if we define causally-correlated flops (so, the seqential, dependent
> FLOPS of a single core, single thread machine is 100%, the FLOPS distributed
> across the globe are 0%, and the distributed cores of a Beowulf working on a
> single problem, but with some wasted effort because not every core can take
> into account every up-to-the-nanosecond current state of every other core)
> is X% for 0 < X < 100. Then, we could define causally-correlated FLOPS per
> cubic Centimter as a unit of information processing density..?  And to find
> the limit you'd have to account for the cc's where the heat goes.
> 
> But I'm no physicist.

Well said, nonetheless.  My only teensy insertion in this otherwise
interesting discussion that is saying it all without me is that current
ML extrapolation is still mostly assuming single threaded von Neummann
computing.  As core count increases, this assumption will become
increasingly false, and using parallel aggregate ips as a measure will
become increasingly misleading as very few applications will be able to
use all the available resources all the time or distribute themselves to
utilize peak.

IMO what we are really waiting for is the next revolution on the CODE
side, and perhaps on the computer architecture side.  This will also IMO
come from AI researchers, not traditional computer science research into
hardware architecture, as hardware design will ultimately have to follow
the development of the science.  Architecturally, processors will have
to migrate from being largely sequential (with pipelines etc, sure, but
sequential) to being largely parallel, possibly "neural", devices.
Their actual engineering will very likely be much more "organic" than
current mask-driven methodology.

The interesting thing about this is the possibility of a nonlinear jump
in ML, or perhaps a complete restructuring of the law to a new form with
a new exponent.  If economics doesn't kill the whole process first.  But
we're a long ways away from the future yet -- I'm still typing at this
computer instead of using the rgbot neural interface I envision and plan
to invent in a few years...

    rgb

> 
> Peter
> 
> On 4/9/09, Prentice Bisbal <prentice at ias.edu> wrote:
>       richard.walsh at comcast.net wrote:
>       >
>       > ----- Original Message -----
>       > From: "Ken Schuster" <ken at kschuster.org>
>       > To: beowulf at beowulf.org
>       > Sent: Wednesday, April 8, 2009 2:29:17 PM GMT -05:00 US/Canada
>       Eastern
>       > Subject: [Beowulf] Moores Law is dying
>       >
>       >>An IBM researcher says Moore's Law is running out of gas. IBM
>       Fellow
>       > Carl Anderson, who
>       >>oversees physical design and tools in its server division,
>       predicted
>       > the end of continued exponential
>       >>scaling down of the size and cost of semiconductors:
>       >
>       >>"There was exponential growth in the railroad industry in the
>       1800s;
>       > there was exponential
>       >>growth in the automobile industry in the 1930s and 1940s; and
>       there was
>       > exponential growth
>       >>in the performance of aircraft until [test pilots reached] the
>       speed of
>       > sound. But eventually
>       >>exponential growth always comes to an end," said Anderson.
>       >
>       > Mmm ... he may be right, but I do not like his historical
>       references
>       > which seem
>       > to conflate engineering and economics.  Better to refer to the
>       > improvement in
>       > magnets or something similar.  But, I like the speed of sound
>       reference
>       > because
>       > it suggests that there is a Moore's Law barrier to be
>       broken.  There is
>       > a lot of
>       > talk about "walls" these days ... the memory wall, the power
>       wall, ...
>       > but we with
>       > respect compute power we have a ways to go before we reach the
>       > Bremermann Limit.
>       >
> 
>
>       This is clearly off-topic, so please feel free to ignore:
>
>       I disagree with the sonic barrier wall analaogy. Is it that
>       clearly
>       technical barrier the slowed down jet research, or did the
>       nuisance of
>       sonic booms to people on the ground just make supersonic R&D
>       less
>       convenient? I've heard that supersonic travel over land is
>       restricted in
>       the US.
> 
>
>       --
>
>       Prentice
>
>       _______________________________________________
>       Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
>       Computing
>       To change your subscription (digest mode or unsubscribe) visit
>       http://www.beowulf.org/mailman/listinfo/beowulf
> 
> 
> 
>

Robert G. Brown                            Phone(cell): 1-919-280-8443
Duke University Physics Dept, Box 90305
Durham, N.C. 27708-0305
Web: http://www.phy.duke.edu/~rgb
Book of Lilith Website: http://www.phy.duke.edu/~rgb/Lilith/Lilith.php
Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977