[Beowulf] First 96-Node Transmeta Desktop Cluster Ships

[Jim Lux]

At 03:35 PM 5/4/2005, Vincent Diepeveen wrote:
>At 11:51 AM 5/4/2005 -0700, Jim Lux wrote:
> >At 10:08 AM 5/4/2005, Vincent Diepeveen wrote:
> >>Yes it is very interesting.
> >>
> >>However their sales price of $100k they initially quoted i found a bit
> >>expensive.
> >
> >I suspect that they picked the price point according to typical "purchasing
> >authority" thresholds.
>
>A wrong way to price things IMHO. In current economic situation no one will
>want to waste money for his company. $100k for such a box, even though it's
>an interesting box, is way too much for the performance it gives.

Not to put too fine a point on it, but any commercial vendor must price on 
the basis of making sales, which in turn depends on how high up the chain 
you have to go for approval.  There is a reason why game consoles run 
around $100.  That price point is well established as a "discretionary 
impulse buy".  You pick your market, pick your price point, and design for 
those points.  You've only got resources to do a very few designs, so you 
have to pick something.  You could argue that maybe $50K or $200K would 
have been a good price, and they might well.

However, $100K is right in line with other expensive capital test equipment 
(like a microwave vector network analyzer or a high performance spectrum 
analyzer).  And it can be justified in exactly the same way as one would 
justify the purchase of a $100K network analyzer.

Just because YOU don't need a VNA worth $100K doesn't mean that there isn't 
a market for such items.



> >>I don't want to be rude, but these processors are quite slow if you just
> >>look to 1 processor and they go back in performance after a while when they
> >>go hot.
> >>
> >>So a fully loaded system will have a far smaller performance in the long
> >>run than it has in the short run.
> >
> >But then, this isn't a device designed to be churning 24/7 serving multiple
> >users.  It's for someone like me to shove under my desk and use in a very
> >"bursty" fashion.  Much like the processor utilization on my desktop
> >computer now, where it runs around 5% most of the time, and bursts to 90%
> >when I fire up Matlab and start crunching.  But then, I go to lunch, and it
> >drops back down.
>
>Give 1 example of a user that has $100k to spend on a machine, without
>actually using it.

I have RF and microwave test equipment in my lab that totals substantially 
more than $100K, and it hardly gets 100% utilization. If nothing else, we 
all go home at night and don't run 3 shifts to keep it busy.  For another, 
there's a standard cycle of design, build, test, evaluate that you go 
through, so the value of that hardware is substantially greater than its 
cost, even though the mean utilization is probably down in the 20% range 
(over just working hours..)  The value is even greater because it's not 
shared among hundreds of users.  I don't have to schedule an appointment to 
go use the VNA when I'm ready to do my next test. I just go and do it.


>Rich Oil-tycoon from texas perhaps?
>If so, mind giving me his adress?



> >
> >
> >>Of course that has its charm too, a system that's fast if you just use it
> >>real short at a "critical" moment.
> >>
> >>So the quoted performance is still impressive then.
> >>
> >>The network is not so impressive what each node is connected to another
> >>node. That is not relevant for embarrassingly parallel software though.
> >>
> >>Relevant is the price per gflop IMHO.
> >
> >Look at TOTAL Cost per USEFUL gflop.
>
>Oh well, for my chess program it's a wasted $100k, so let's not get in that
>discussion.
>
>The network is way too weak. one way ping pong latencies between the
>processors i didn't see published, i suspect around 200 microseconds or so?
>
>Because for all the shared memory software it's not a useful machine, just
>for embarrassingly parallel software, in general that means that it can run
>toy programs.

Given that there's a great interest in cluster computer software, one might 
hope that new software will take advantage of (or at least be insensitive 
to) cluster peculiar characteristics (i.e. high latency comm).

For that matter, many tasks that I do fall in the EP category. Monte Carlo 
circuit design and analysis is a good example.  You build the circuit, then 
try randomly varying component values to ensure that it will work over 
manufacturing variability.  highly parallel, low comm need, computationally 
intensive.


> >
> >>For a NEW product that tries to fight itself into the market, you must
> >>simply be factors cheaper than the competition. $100k i find a tad
>expensive.
> >
> >Or, find a new and different market that has different requirements, not
>
>
>Government organisations will not soon buy these boxes, apart from that i
>do not know a single government body that would get a budget for a $100k
>computer, does the processor have ECC in the L1 and L2 caches?

The user of a computer shouldn't care whether it has no, one, two, or a 
hundred cache levels.  All they should care about is "does it make my job 
easier/faster".

There are an awful lot of people paid a fair sum of money to do things not 
computer related (i.e. designing wireless handsets and cell phones, doing 
mechanical design of automotive components, etc.) who can use more 
processor crunch, IF it's easy to use and doesn't require a lot of cluster 
specific knowledge (i.e. turnkey software). That IS the market that Orion 
is going after (I assume)..   Secretaries in 1985 did not care whether the 
PC had 64, 128, or 256K of RAM.  They found it easier to cut and paste with 
a cursor than with scissors and paste, and that's where the value is.


>I agree, if that is the case, that it is a ridicioulous question,
>but they are requiring it for computations.
>
> >currently well addressed by the existing marketplace.  This product will
> >NEVER compete against a rack of 4U chassis with a team of gradstudents
> >installing and maintaining it for (virtually) free.  Nor will it compete
> >against any "home built" system, where labor is free.  Nor will it compete
> >against the 1024 processor dreadnought class clusters.
>
>It's in the same price league like a home build dreadnought.

Which is a pretty impressive feat, since commercially built dreadnoughts 
are a heck of a lot more than $100K.  FWIW, I don't think an ambitious home 
builder could build a thousand processor cluster for <$100/processor, even 
with free labor, except perhaps by the "Stone Soupercomputer" approach.


> >
> >They'll not sell it on dollars/GFLOP.  They'll sell it on the basis of
> >increased productivity of expensive people.  The structural analyst with a
> >fully burdened cost of $250K/yr who becomes 15% more productive, for
> >instance, will pay for it in the usual 3 year amortization
> >interval.  Justifying that cost/benefit will be the real challenge, but
> >it's no different (conceptually) than justifying a $5K wordprocessor for a
> >$15K/yr typist. (1985 GS-3 Typist II.. probably after you add burden,
> >around $30K/yr cost)
>
>Ah, again you're showing how commercial government officials are.

Not just government. Industry. There's nobody throwing money at people to 
just have fun.  Somewhere down the line there has to be a cost benefit 
tradeoff, whether its viewed in terms of "increased human knowledge" (i.e. 
NASA space program) or in terms of Return on Shareholder Equity (i.e. most 
corporations).


>Believe me, no objective economic measurement will cause a company to buy
>this type of machines, as there are cheaper alternatives eating less power,
>producing more gflops when you need it.

Orion is betting you're wrong.

>Please note that nowadays your salaries for programmers, world wide seen,
>are not true at all.
>
>First of all majority of software does NOT get made in USA.
>
>Secondly in this country, i would really be HAPPY with a contract job to
>parallellize software (or whatever algorithmic software) around $30k.
>
>Because of the low dollar it still looks like a lot (1.28 dollar = 1 euro)
>30k euro is about what a programmer earns here.
>very good project leaders of programming teams are peeking at 42k euro a
>year. Usually 10% of that additional to bonus.

Perhaps in India, a company would not find a Orion system a good 
investment, against a $20K/yr salary.  However, skilled engineers in the US 
do have salaries in the $80K-$100K/yr range, which works out to a burdened 
cost (heat light, rent, benefits, vacation, etc.) around 
$200-250K/yr.  While some tasks get outsourced, not all do, and for those 
that remain here, saving 10% by improving productivity is a "good thing"




> >Here's a quote from a 1989 business magazine:"
> >When all is said and done, however, forget about features; forget about
> >price. Instead, choose the program that you can get help with at midnight
> >on a Sunday when you have a proposal due at eight o'clock on Monday morning.
> >"
> >Another report from that era mentions that factories capitalize their
> >workers (in 1981) at $25K/worker (that is, they invested $25K in plant and
> >machinery for that worker), while clerical staff was capitalized at $2.9K.
>
>You left businessworld in 1989 and started a job at government?

No, I'm attempting to draw the parallel between the introduction of PCs and 
dedicated word processors in the context of timesharing and mainframes (mid 
80s) to cluster computing today (turnkey cluster appliances vs racks of CPUS)


> >
> >Au contraire...  The problem with power isn't the electricity cost.  It's
> >the removal of the heat and other infrastructure costs.  Those 10 quad dual
>
>Houston, we have a heat problem outside!
>
>(loud laughter), it's nearly summer here now. It's 10C outside. Real nice
>temperature. If i open the window, all heat here is directly gone from 3
>dual machines, each having a 460 - 550 watt power supply. In the winter i
>never open the window and turn the additional central heating a bit lower.

Try that in Los Angeles (or Houston, for that matter)...



> >core opterons will probably fill a rack and have a mighty roar of cooling
> >fans, and I won't be able to plug it in in my office built back in the
> >60's.  (assuming I have ear plugs)
>
>Basically i do not know any system >= 4 cpu's that's real quiet.
>Well you're deaf anyway, this orion system produces 50-55 decibel,
>that's ear deafening.

50-55 dB is about like a standard PC.

> >

James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875