[Beowulf] MS HPC... Oh dear...

[Robert G. Brown]

On Mon, 12 Jun 2006, J.A.Delcorso at larc.nasa.gov wrote:

> Robert,
>
> For the most part I agree with a lot of what you've put out on the
> Beowulf list, but this last response kind of struck a cord.  I'm
> positive you have much more experience in these matters, so if I'm
> way off in my comments please feel free to correct me.
>
>> Remember also that the "advantages" of being able to run MS-only are
>> largely illusory, since MCSEs are largely clueless about advanced
>> networking and parallel computation and parallel scaling and MPI and...
>
> Dangerous over-generalization... Why are all folks who *might* maintain
> a *chuckle* Windows cluster immediately MCSEs?  We've got a fairly large
> cohort of *nix admins who've developed and support our beowulfs.  They
> are generally also the folks who maintain our MS pc's at the same time.
> These are also the folks tasked with looking into these (MS cluster)
> 'new' type of systems.
>
> They _are_ very bright and know their stuff...  The problem isn't the
> admins in this case, it's the perception from 'on high' that MS is
> the best & only operating system (or at least the one that management
> uses & logically if they use it we should all use it... or some such
> nonsense...)

Oh, fine, sure, right.  It WAS a RANT, remember:-) And a fairly openly
anti-Microsoft rant at that...;-)

I actually agree with this, but.  The but is that sites that have your
mix are going to be unlikely to pay money to get "Windows HPC".  What
for?  Why buy a cow when the milk is free?

WinXX clusters would have to produce a really tremendous advantage in
application, and I just don't see it ever doing so.  Joe seems to think
that they'll get traction by defining an MPI ABI -- I think that they're
market presence will simply fracture the existing efforts to define one
even more.  It isn't like a WinXX binary is going to run on Linux,
right?  Windows AND MPI will be just like Linux AND any existing MPI, at
best.  And besides, I personally think that it is API that counts, not
ABI, except maybe possibly at the hardware driver level.  Is anyone
forseeing Myricom abandoning the Linux market?  Quadrics?  Infiniband?
Yeah, right...

And the whole point of MPI in the first place was to precisely counter
any effort by a single company to introduce proprietary crap that adds
to the cost of software ports or maintenance.  Does anyone think that
the entire world of parallel programmers has suddenly infarcted their
brains and forgotten that so that they'll buy into an MS attempt to
co-opt MPI?

>> so WinXX clusters will face precisely the same support and programming
>> challenges that Linux clusters face without the rather huge base of
>> coders, the beowulf list for distributed (free) support, a wide
>> selection of consultants and turnkey vendors, magazine columns and
>> websites such as the Monkey.  This is an "if we build it, they will
>> come" moment for MS.
>
>
> Agreed... the problem IMHO is that when MS puts out something like this
> the immediate perception is that there is support and (though this might
> be a dangerous over generalization) companies developing commercial
> applications tend to fall in line behind the idea... which leads to the
> snowball effect...  So yes, they will face the same support and
> programming challenges, but they won't "appear" to be challenges because
> (going back to perception) Microsoft is putting it's weight behind
> the development.  I don't doubt that within a year we will see some
> kind of "Microsoft Visual Studio Professional Parallel Programming"
> software out on the market (if it's not already).  There will be a big
> to-do about it, and it will get media attention, which of course gets
> back to the folks who make decisions 'on high', and now we're back in
> the cycle of MS...

Well, for a variety of reasons I think that -- although this is
doubtless their "plan" -- this just won't be happening.  Parallel code
development hasn't exactly dominated the entreprenueurial market, hey?
It's difficult, expensive, and there aren't that many applications out
there that anybody will pay for.  In my opinion, of course.  There will
be some, of course, but there Windows clusters will have to pass the
same rigorous requirements for good cost-benefit that the various
choices of linux clusters do now.  Cluster scaling is far and away
dominated by HARDWARE resources and scaling, not software.  So it will
come right down to trading cluster nodes for Windows licenses unless
they drop the cost of the latter to literally nothing.  And if they do
that, what's the point?

>
>
>> Compared to linux -- the entire OS and all
>> these apps were FREE, right?  One can install an entire cluster for the
>> cost of setting up a PXE server/repo, and even boot and run the whole
>> thing diskless, in as little as one day.
>
>
> Preaching to the choir...  *KEEP PREACHING, the beer is on it's way*

Best offer I've had in days.

>
>
>> Only people who really don't
>> give a rodent's furry behind about money will be willing to spend a ton
>> of money for a product where it takes days for your cluster-ignorant
>> MCSEs just to figure out the licensing arrangements for the nodes (and
>> to learn what a node IS).
>
>
> Again... dangerous over generalization IMHO.  Perception will be something
> closer to "hrm... $8,000 and less headache with MS than with going to
> a linux system... It's worth it."

Why less headache?  Let's see.

   Paying money = Bad (and $8000 is going to be for how many nodes?  How
will costs scale?  What do you get for this?)

   Ease of installation?  Are they claiming to beat "boot the node and
wait for it to install" or even "boot the node diskless and wait for it
to boot"?  For $8000 I'll cheerily make you a linux server that will
boot your cluster either way and give you change back -- or better yet
buy myself more beer.  And that includes the cost of the hardware --
damn good hardware.

   Software maintenance?  Competing with yum and the repo mirror tree (as
just one example)?

   Human costs?  Right now there are numerous clusters with a single
pro-grade admin caring for hundreds of nodes, as many as perhaps 500 or
even more in the extreme cases.  The limiting factor on cluster size
that can be managed by a single human is HARDWARE reliability, and with
very reliable hardware backed by a quality vendor and onsite service
etc., that number is very large indeed.  Updates are fully automated.
Configuration is trivial.  Ability to monitor cluster resources of all
sorts is readily available (hmmm, just how easy is it to figure out what
the hell Windows is DOING at any given time -- hmmm, not too easy...).

   File sharing?  Oh, wait, isn't that all about Windows Server?  Have
they suddenly changed the scaling of that product so a single server can
actually manage more than a half dozen clients?  Wait, lessee -- now
they claim that the server running on a X64 server can handle maybe
TWICE that.  Wow!  One rather expects that their cluster product is
carefully designed to either avoid remote filesystem access, or use a
single Samba server (right!;-) or heck, just use NFS.

   And the list goes on.  Not to mention the "obvious" point that it is
EXPENSIVE to port software to a new platform.  Nobody will do this
unless there are clear and unmistakable benefits, not just a much-hyped
appearance of Microsoft in a market they've wisely avoided for years.

I'm still waiting to see where anybody sane would be excited by the
appearance of Windows HPC.  Microsoft even runs the risk that if they
make it TOO efficient, it will close out Windows SERVER as a useful
product.  I therefore expect it to be scientifically crippled in that
regard.  Right now Server lets them just about exactly double their
operating system profits per seat in a client/server environment.  If
they make any part of "Windows HPC" capable of provide services more
cheaply per "node", gee, people might stop buying Windows Server and run
their LANs on top of Windows HPC.

Does anyone seriously think that a cluster could rationally be run on
top of Windows Server as things currently stand?  That's the scaling
benchmark that one part of the company will be fighting hard to
preserve, one that is worth 100x what the cluster market is worth.

> I remember a couple years back when folks were saying that MS has little
> or no interest in expanding into the cluster market... yet here they
> are knocking on the door.

Naa, they've been trying to get into it for years.  Their basic product
simply hasn't been CAPABLE of supporting it in the past in any sort of
credible way.  I'll wait with great interest to see if their current one
can.  If their clever idea is to run Windows Remote Desktop across a
cluster of nodes for management purposes, well, pardon me while I go
have a hysterical fit ROTFL.  If they support something ELSE, well, how
long will it's Server product last?  It would cost them billions.

This is why I think that it is all about something else.  Suturing a
bleeding wound in public relations, supplying a limited market for small
clusters, supplying an expensive and profitable model for turnkey
bioinformatics clusters.  Don't look for them in places where people
have to write their own code, or use a widely shared open source code
base.  And it is not without its risks.  If they fail, their bulletproof
image will be severely shaken.  If they succeed, they risk their client
server profit margins, as a cluster ain't nothing but a fancy client
server model.

>> to buy it, although they could lose money on this for years and still
>> fund it just to be able to claim market presence and not think twice
>> about it.
>
>
> Yeah, and as I alluded earlier... Perception for many is reality.
> Given enough time people will believe that MS is a viable solution,
> just a matter of biding their time and waiting...
>
> Looking into my crystal ball it doesn't take a genuis to see this.
> MS will wise up eventually, and change their marketing strategy so
> that the cost of the parallel environment is negligible, and the
> real money will be in the development tools.  At that point (I'm
> assuming here) MS will have a couple applications running they
> they'll point to as examples of how parallel systems work with
> MS and "oh look, you can use MS Office also! Isn't that fun?!".
> Again, pushing the perception...

As I said, ROTFL.  That works fine for numb-nuts spending $500.  It
doesn't work that well for corporate or government decision makers
controlling the disposal of $500,000, where the question is whether it
buys (say) 2000 Linux nodes or 1000 Microsoft HPC nodes.  Somebody's
going to have to answer some cold hard questions about just how the 1000
nodes are going to do the work of 2000.  Where I personally think that
it will be a cold day in hell when Microsoft will scale as well as Linux
node for node in anything like a realistic nontrivial application space.
In linux you have direct access to low level internals and the ability
to fix things.  In the case of Microsoft HPC, you're going to get what
MS gives you and like it -- or not.

>> maybe they'll get some gaming companies to use
>> their platform to manage multitasking on a dual-core dual-cpu box to
>> improve game performance or the like (again on a shrink-wrap basis).
>
> UGH... lets hope not... I vaguely remember that the gaming industry
> was a fairly key player in the past, pushing both hardware and software
> solutions.  Yet another 'advantage' of MS clustering if they get their
> hooks into the gaming industry... a bunch of kids wanting to develop
> parallel gaming applications... talk about your snowball effects if that
> works out.
>
>> So, naaaaa, not likely to be a popular development platform for real
>> researcher's writing their own code or using open source code.
>> Commercial only.
>
>
> I wish that were true.  The reality seems to be somewhat different than
> what makes logical sense.
>
> Any more, the folks coming out of college have virtually no *nix
> experience.  Universities are pushing Windows OS and development
> like there's no tomorrow.  While there are many instances of universities

Not here.  Not anywhere I know of.  Java, yes.  Web stuff, yes.
Honestly, Universities aren't even pushing compilers and real
programming that much any more from what I see.

> pushing equally hard on *nix systems, the vast majority of engineers/CS
> folks that we're seeing out of college are MS developers.
> Personally I've been through more Windows vs Linux benefits/drawback
> discussions with these folks in the last year than in the previous
> 5; and I only see the trend getting worse.  The main problem is that
> when these folks talk, they have no understanding of what *nix is
> because they've never used it, never seen it, and have been spoon
> fed the MS mantra for years in college, nor do they care to learn...
> to make matters worse, these are the folks who are growing in number
> around me, and what I'm seeing in many of the companies we work with.
> These folks *DO* influence decisions, even if they're more 'costly'
> decisions.  The arguement is simple and to managment, a little more
> money to make life easier is worth it (remember _perception_ here).
>
> Imagine if MS wises up and starts providing their cluster solution
> to the same colleges that are pushing the MS development environments...
>
> Here, perception is reality (sadly enough), and more recently I've
> been required to switch to Windows systems because it fits into the
> powerpoint-lifestyle of the environment. (required == mandated)

Well, we'll see.  The real question will likely be how much GUI
integration is good for in parallel software development, as that's what
MS's tools are good for (if anything).  Alas, I don't see that as being
worth a damn in parallel software engineering.  But what do I know...;-)

It will be nice for shrink-wrap applications, maybe, though.  As I said,
commercial software development, for that very, very limited market.
Real researchers would view a nice GUI as being fairly low on the list
of priorities in the grand scheme of things compared to things like
optimizing their parallel scaling, something that a GUI isn't going to
help with much.

    rgb

>
>
> Just my 2cents...
>
> Joe
>

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu