[Beowulf] MS HPC... Oh dear...

[Vincent Diepeveen]

Not that i'm a big m$ fan in this, but your conclusion that:
  a) highend is easy to use
  b) highend is very cheap

I have to disagree in both cases.

First of all if i ask my 76+ year old neighbour who is a famous writer
and she has been using computers to write her books already for longer
than i live, at computers, whether she can simply *turn on* the power
at my cluster, that's already too complicated.

Let's not talk about installing it and that to use it you need to know all
kind of freak linux commands, not to mention knowing pdsh.

Linux is NOT easy to use.

Now let's move to price.

If i produce 1 cdrom manually the cost of that is my own labour,
which ain't cheap in my eyes + cost of 1 cdrom.

If i produce 500 cdroms, i'm paying about 2 euro a cdrom,
including printing and packaging in a DVD case.

If i produce 5000 cdroms, i'm paying about 0.5-0.8 euro a cdrom.

The price goes down dramatically in short when the quantity goes up.

If i order 1 A64 dual core, i pay nearly 1000 euro including all kind of
funky stuff. If i get a 'bargain' i'm ready for around 500 euro.

If you order some 'highend' machine, a dual opteron dual core, you're
paying not LESS than 2 * 1000 euro, but a multiple of it.

If you order a CLUSTER you're soon paying 5000-6000 dollar a node!

If you expand to a big supercomputer, of course with a good network,
price may even go up.

Highend when scaling up is NOT getting cheaper a unit any soon.

Now we didn't talk about the software to execute at it yet.

Last week i received several times the question whether some software of a 
college of
mine, whether i could simply 'get it parallel to work' for the person in 
question at his new
'quad opteron dual core'.

How about a cluster now?

How do you run your favourite racing game faster at it?

Now about your misplaced 'market growth' terminology. That's not exactly a 
good way to put it.

There is a lot of factors to consider. First of all is 'inflation'.
20% growth doesn't say anything if there was 20% inflation lately.

Secondly is the growth of internet, which screams for 'servers' for 
companies.

Third is the total market that has opened for IT, with a lot of nations 
growing in all directions.
China and India everyone will mention, but there is another huge growth in a 
lot of European nations
that recently became members of EU and some that are on the brink of 
joining.

Where everyone sees India and China, if you add up total growth in Europe, 
then
Europe is growing to a major giant in world economics.

Fourth is the lowering of the definition of highend. Not too long ago 
anything with x86 processors wasn't called highend.
Now if i buy a pc for 500 euro, compile apache at it, i can already call it 
a server and qualify for 'highend' ?

On the other hand my windows PC which is a dual opteron dual core 2.4ghz 
where i run especially my chessprogram at,
that isn't called highend?

Sir, it's having a psu of 680 watt and a raid10 array of 4x SATA drives!

A big airflow nonstop gets put through the entire case which i can't call 
'quiet' as a result of that.

Fifth is the hard fact that more and more of todays generation knows how to 
use the difficult to use linux operating system
and simply have a lot of knowledge on how to use software in general.

The definition in statistics on population numbers in cities calls that 
workers group 'nerds'. That group is 5% nowadays.

So against that a 20% growth claim honestely isn't impressing me much.

In fact i'm not amazed if you soon call every windows machine a server, 
because it runs 'services' that are clear server services.

Vincent

----- Original Message ----- 
From: "Joe Landman" <landman at scalableinformatics.com>
To: "Gerry Creager N5JXS" <gerry.creager at tamu.edu>
Cc: <beowulf at beowulf.org>; "SIM DOG" <steve_heaton at iinet.net.au>
Sent: Monday, June 12, 2006 5:02 AM
Subject: Re: [Beowulf] MS HPC... Oh dear...


> (sorry in advance for the length)
>
> Gerry Creager N5JXS wrote:
>> Ooh!  ISO-9002 Buzzword Compliant marketing.
>
> Hmmm.... not astroturfing here.  I opine on http://scalability.org/?p=69
> .  A few others have linked to this, so we are getting some traffic.
>
> Specifically, I am of the opinion that the sentence "but until now it
> has been too expensive and too difficult for many people to use
> effectively" is factually wrong.   The reasoning is very simple, and
> borne out by existing data.
>
> If HPC has been both too expensive AND too difficult to use, then why is
> it as a market growing at 20+% per year?  Moreover, as IDC points out
> that Linux clusters are driving the growth in the HPC market, with this
> segment currently at about 1/2 of the $9B market and growing at 60+% per
> year for the past 3 years ...
>
> Short version: it is neither too expensive, nor too hard, as people are
> doing it effectively now, and have been for the last 5+ years, with the
> growth showing up on the radar 3 years ago.
>
> If the statement of too expensive could be applied to Linux clusters,
> then this fails to explain the observation of the growth.  Last I
> checked, real measurement trumps hypothesis.  These aren't windows
> clusters that are growing, these are Linux clusters.  These aren't unix
> boxen.  They are Linux clusters.  There must be a reason for this.
>
> If the statement of too hard could be applied to the market, one would
> need to ask exactly what people were buying that was not too hard which
> is generating all that growth.  Since we know the answer (linux
> clusters), they must not be too hard to use.  The systems we put
> together for our customers who don't care what is under the hood looks a
> great deal like a large windows disk (or disks) and a web page.  Those
> who care about the details prefer the command line.
>
> All this said, and not to disagree with Doug Eadline and others on the
> technical details, I do think Microsoft has something to offer here, but
> I think they need to work within the existing community, and not dismiss
> it out of hand.  The latter is the sense I am getting out of the
> marketing.  The folks from Microsoft I have spoken with, Kyril and
> Patrick, seem to be quite interested in doing the right thing, though
> with a decidedly Microsoft spin.  The problem I see is that the spin and
> some of the core assumptions are, IMO, incorrect.  One expects marketing
> to be so.  Building a go-to-market strategy upon the basis of core
> assumptions that are not in line with the market reality is dangerous,
> even for an infinitely deep pocketed corporation.  All I suggest is
> appropriate debunking of the marketing, and drilling into the
> technological core of what they are going to market.
>
> They do have a number of very hard hills to climb, specifically pricing
> compared to competitors, technological feature lists, interoperability,
> security, and stability.  Most of these are going to work against it.
> It would be unwise to count them out of the game though.  Anyone
> remember or still use Lotus 123?  Wordperfect?  May take them a while,
> and they are persistent.  With very deep pockets, lots of patience, and
> the ability to purchase talent.
>
> Linux was able to effectively kill Unix by presenting a single API to
> write to, a simple stack to deal with, a much larger potential installed
> base, a lower cost of acquisition.  Microsoft has learned from this.
> Assume that this is their direction.  The arguments they presented to me
> involved driving a wedge between various linux distros, and painting the
> Linux scene in a similar manner.  Their MPI argument (to many stacks)
> was not a good one, as the same problem exists on windows.  But the
> point is one that I and many others have complained about at some point
> in time or the other.  You have different MPI stacks which are binary
> incompatible.  Which means if the PathScale folks came out with a new
> hardware device to accelerate networking for folks like LSTC, then the
> LSTC folks have to relink their app against the new stack.  Which is
> exactly what happened.  While some folks here defend this, I want to
> note that end users don't give a rip about that.  They want the new
> fangled hardware to work.  Right away.  Without a rebuild of the app.
> So do the vendors.
>
> ISVs don't want a rotating collection of MPI stacks, but one to work
> with.  One API.  Each app now needs to decide how many MPI stacks to
> support on each ABI.  You have IA64 with several, AMD64 with several,
> ia32 with several, ...  Each stack adds cost/complexity to them.  Again,
> this is something I have argued for a while.  The ISVs as well.  They
> don't want to support Joe's MPI stack, they want one MPI stack per ABI
> (even better would be one MPI stack and one ABI, but we are not there
> yet).  MPICH runs pretty much everywhere.  LAM (which I like a little
> better) runs fewer places (never been able to get it to work the right
> way under Cygwin).  I can live with MPICH.  The problem though is that
> we have to relink the application for each new networking advance.  And
> customers don't like that.
>
> What Microsoft will do is to take away as much of this as they can.  I
> haven't seen it yet, but I believe they will offer MPICH as a DLL, so if
> PathScale wants to work along side some other device, you can select
> this at runtime, and just have it work.  This is a nice idea.
>
> Schedulers are another area, but my impression from speaking with them
> is that they haven't looked at the market carefully enough yet, or some
> non-business reasons got in the way of them exploring whats out there.
>
> The idea is that they have a good story in some parts, and a very weak
> story in others.  Assume they will improve the weak points.  It would
> make more sense to engage them so that they improve the weak points
> along reasonable lines.  I would rather have them fit in, than try to
> overtake, as the latter will just piss off the customers when they
> realize what they want to do is not possible with their shiny new WCC,
> or they suffer far worse performance than the folks with the Linux
> cluster due to all those corporate mandated firewalls and copies of
> Norton running.
>
> Joe
>
> -- 
> Joseph Landman, Ph.D
> Founder and CEO
> Scalable Informatics LLC,
> email: landman at scalableinformatics.com
> web  : http://www.scalableinformatics.com
> phone: +1 734 786 8423
> fax  : +1 734 786 8452 or +1 866 888 3112
> cell : +1 734 612 4615
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf
>