[Beowulf] MS HPC... Oh dear...

[William Harman]

Joe, rgb, Jim, & all

I think that the point is where they will position their product, and IMHO
it is what we used to call the scientific workstation, a decade or two ago.
They are going after the personal supercomputer, with up to 96 CPU's under
the desk today, and many more tomorrow, as motherboards hold more CPU's and
more cores.  The engineer, designer or scientist is not concerned with
what's under the hood, only that his work gets done, in a reasonably
timeframe and with an interface that he is familiar with and can control,
i.e. Windows, and an output that he can use, usually programs that have been
written for Windows, such as visualizations, as mentioned in an earlier
post.  He will run commercial software and his financial guys are quite
happy to amortize the cost over 3 years, when he will be allowed to make an
upgrade.

Remember that only 4 years ago a 96-node cluster was a major purchase for
most companies.  

It's mass marketing of personal supercomputers.....and the commercial
software vendors are onboard with M$$$$$ (Look at who has partnered with
them already).  I have no doubt that in 2 years they will have 20%-25% of
what IDC calls the HPC market, and they may do it without even having a
system in the Top500.

At least that's the way I see it.

Bill Harman,


-----Original Message-----
From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On
Behalf Of Joe Landman
Sent: Monday, June 12, 2006 7:36 PM
To: Robert G. Brown
Cc: beowulf at beowulf.org; J.A.Delcorso at larc.nasa.gov
Subject: Re: [Beowulf] MS HPC... Oh dear...



Robert G. Brown wrote:

> WinXX clusters would have to produce a really tremendous advantage in 
> application, and I just don't see it ever doing so.  Joe seems to 
> think that they'll get traction by defining an MPI ABI -- I think that 
> they're

Actually I think they will get traction from their infinite marketing
dollars, some high profile wins, and pressure upon the C-lettered people to
enforce platform monoculture.  I doubt it will lower their support costs
(will raise them quite a bit IMO), but thats what will be fed to the C-level
folks.

My point about MPI is that they are going to do what they can to make it
easy, and make it just work.  While I like the concept of making it just
work, there is a cost to participating in that model that I have not
evaluated yet.

> market presence will simply fracture the existing efforts to define 
> one even more.  It isn't like a WinXX binary is going to run on Linux, 
> right?  Windows AND MPI will be just like Linux AND any existing MPI, 
> at best.  And besides, I personally think that it is API that counts, 
> not ABI, except maybe possibly at the hardware driver level.  Is 
> anyone

That was my point.  I would like to see the API/ABI to the point where, when
Greg's company comes out with Infinipath 2007++ with ESP technology to send
the packet your code was thinking about sending, but never got around to
sending, that we don't need to relink code to use it.  Drop the driver in,
have the ABI do its magic, with some possibly environment tunable
parameters, and start the computing engines.  Pushes more work to NIC/data
pump vendors, but it isn't so terrible, and it makes other peoples lives
soooooo much better (wrestling with an obscure version of LAM that doesn't
cause segfaults with LS Dyna is *not fun*(TM).

> forseeing Myricom abandoning the Linux market?  Quadrics?  Infiniband?
> Yeah, right...
> 
> And the whole point of MPI in the first place was to precisely counter 
> any effort by a single company to introduce proprietary crap that adds 
> to the cost of software ports or maintenance.  Does anyone think that

Unfortunately, we are at the point where each vendor has to ship (most
of) the MPI they built with in order to make sure the customer can run it,
and worse, the new fangled hardware (see the imagined Infinipath
2007++ with ESP technology that can get negative latencies) doesn't work
with it, so ...

This frustrates software builders, and end users.  My point is that there is
a better way, and Greg indicated that he had supported/proposed it.

Windows could get traction by making this stuff easier.  They have done it
before with other things.  Note:  easier != better in all cases.

[...]

> choices of linux clusters do now.  Cluster scaling is far and away 
> dominated by HARDWARE resources and scaling, not software.  So it will

Hmmm.... most of the apps I have seen are software bound at some point.
 Some scale really well, but those are rare.  16-32 way runs are fairly
typical at customer sites.  A few do more, many do less.

> come right down to trading cluster nodes for Windows licenses unless 
> they drop the cost of the latter to literally nothing.  And if they do 
> that, what's the point?

The quality of the software is what dominates performance, and the price of
the software limits practical scalability.  At 10k$/node, the software will
far outstrip the hardware in terms of price scaling.

That said, the cost of the windows solution will increase the cost of the
cluster in a critical price sensitive area.  Given the hardware margins are
very low in the area that WCC targets, there is very little room to
accommodate this extra cost.  Assume hardware costs of $2500/node roughly.
16 nodes (32 CPUs) would cost 40k$. Now add the 8k$ that Microsoft wants
from this.  That is a 20% increase in cost.  What does the customer get for
that extra 20%?  Will the software cost 20% less per node?  Not likely.
Will the hardware vendors operating in the 3-8% margin region take 20% off?
Heck no.

Here is where the CBA makes sense to do.  What is the value of the extra 20%
as compared to the alternative solution?  What do you get for it?
Exactly what pain is the extra 20% solving?  Note that this is not really
the case, as you need a $50 version of Norton per machine, so thats another
$800.  And how long will these machines be down on/after patch tuesday?

So the question is, is all the extra cost worth having an MPI that just
works?  Is MPI that painful (ok it can be really annoying sometimes when you
are debugging a problem, but usually, once you fix it, it stays fixed).  Is
all that cost worth having the same exact administration model for the
laptop as for the research/engineering supercomputer?  I am not convinced.

[...]

>> closer to "hrm... $8,000 and less headache with MS than with going to 
>> a linux system... It's worth it."
> 
> Why less headache?  Let's see.

I disagree that it is less headache to go with the monoculture.  Show me a
linux admin who is pulling out their hair on patch tuesday through the
subsequent patch tuesday due to issues that the last patch bolus caused.
 Most of the linux admins I know at fortune 500's get pulled over to help
out with the windows side before during and after patch tuesday, as the
admins simply cannot handle the number of problems that arise.  I called
this out as an example of how I believe Microsoft misunderstood its market.

Moreover, on every node you need an antivirus/firewall.  Corporate mandates
that for every windows PC regardless of function.  Have seen lots of that as
well.

My point is that unless they did something spectacular so that WCC is
virus/work repellent, I think the problem is only going to be exacerbated.
Worse, there are companies for which their computing cluster machines being
down for time scales on the order of hours could mean significant money
lost.  Many of them now have farms of Linux clusters, and I would be
surprised to see them adopt a new platform.
Downtime costs real money.   Patching and protecting costs real money.
Adds to admin overhead, reduces duty cycle.

[...]

>   Software maintenance?  Competing with yum and the repo mirror tree 
> (as just one example)?

OT:  I am happy to report that SuSE 10.1 has a working yum (not the one I
hacked together for 10.0 and 9.3), and that I have created a repo for it,
and will be doing some warewulf vnfs test ports/builds (woot!).
Our major port of the ww-2.6.2 is on our download web site in src.rpm and
x86_64.rpm form.

Note:  with this, I could (easily) build VMware players, run Linux diskless
and VMWare running off a disk image hosted locally with backup copies on a
remote server somewhere.  This would be a "workable" windows cluster model.
You wouldn't have to run an antivirus, or a firewall.
Yes you pay the cost of performance in virtualization.  The ease of admin
can't be beat though.  A windows node gets hosed, and you kill the VMware
player, copy an up-to-date version of the disk image over, and reboot
VMware.  Could even do it while the errant VMware is still running, as long
as you use a different file name for the disk image.

[...]

>   And the list goes on.  Not to mention the "obvious" point that it is 
> EXPENSIVE to port software to a new platform.  Nobody will do this 
> unless there are clear and unmistakable benefits, not just a 
> much-hyped appearance of Microsoft in a market they've wisely avoided for
years.

If the porting environment is made very easy to port to (e.g. little effort,
codes run with a simple recompile) then I expect to see more ports.

[...]

> This is why I think that it is all about something else.  Suturing a 
> bleeding wound in public relations, supplying a limited market for 
> small clusters, supplying an expensive and profitable model for 
> turnkey bioinformatics clusters.  Don't look for them in places where 
> people

Not sure I agree with this (expensive/profitable model for informatics
clusters).

> have to write their own code, or use a widely shared open source code 
> base.  And it is not without its risks.  If they fail, their 
> bulletproof image will be severely shaken.  If they succeed, they risk 
> their client server profit margins, as a cluster ain't nothing but a 
> fancy client server model.

Hmmm ...  see http://en.wikipedia.org/wiki/Creative_destruction and Clayton
Christensen's Innovator's Dilemma
http://www.claytonchristensen.com/publications.html .  If you don't
cannibalize your own market, then your competitor surely will.  Linux is
eating a growing portion of Microsoft's lunch.  Microsoft needs to work out
how to respond.  This is IMO one of the responses.

[...]

> As I said, ROTFL.  That works fine for numb-nuts spending $500.  It 
> doesn't work that well for corporate or government decision makers 
> controlling the disposal of $500,000, where the question is whether it 
> buys (say) 2000 Linux nodes or 1000 Microsoft HPC nodes.  Somebody's

Note to self:  Find out who the heck is selling Opteron servers for
$250/node (see RGB's math above).  :)

[....]

>> Any more, the folks coming out of college have virtually no *nix 
>> experience.  Universities are pushing Windows OS and development like 
>> there's no tomorrow.  While there are many instances of universities
> 
> Not here.  Not anywhere I know of.  Java, yes.  Web stuff, yes.
> Honestly, Universities aren't even pushing compilers and real 
> programming that much any more from what I see.

[switching hats for a moment]  When I taught a class this past year at my
alma mater on HPC, a single student in the class had *nix experience.
 Few had programming experience outside of Matlab or C++.  Fortran?
They don't do no steenkeen 52 year old computer languages ...  It is so
yesterday...

Most did Java.  All did windows.  The CLI was a massive shock to their
systems.  That you could work on your assignments from home and run them on
a machine miles away was either a pleasant or scary surprise.  These are the
scientists and computer scientists of tomorrow.  All they know is visual
studio, java, and other similar things.

Joe

--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org To change your subscription
(digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf