[Beowulf] MS HPC... Oh dear...

[Jim Lux]

At 02:40 PM 6/12/2006, Robert G. Brown wrote:

>Note to all others:  The following is a patented rgb rant with no
>otherwise meaningful content.  Hey, it has been a while...:-)

OK... it's been boring on the list recently. I'll take up the cudgels for 
Microsoft, at least as I interpret their viewpoint. I must confess, my take 
probably differs substantially from what MS thinks, but, oddly (or perhaps 
not) MS doesn't participate much in this list...



>Anyway, feel free to hit the "d" key now and skip it.
>
>
>To turn your own observation around, "Competition is hardly possible
>against that from software viewpoint"... especially since Linux "HPC"
>isn't being given away free "initially" -- it is guaranteed to be free
>free by the GPL and other open licenses used throughout pretty much
>forever.  This isn't a case where Microsoft can, um, "undersell" its
>competitors short of giving them money along with their product -- which
>I fully expect them to do, by the way, especially initially -- and any
>effort they expend in this direction is just money pissed away except in
>very limited and specialized commercial markets.


I wouldn't even go so far as to say "short of".. MS can and has funded 
research projects with cash as well as product.  No reason why they 
shouldn't.  Being perceived as a bastion of the evil software empire, they 
might find it harder to find volunteers to toil on their behalf, so they 
must purchase theit toilers.  No matter, they've got lots of excess cash to 
spend on such things, even if just out of idle corporate curiosity.



>You also missed another of my points.  It has been possible to write
>parallel software that runs on Windows boxes since maybe 1993 (can't
>recall the exact date that somebody did the Windows port of PVM, but I
>vaguely recall seeing Windows ifdefs in the source about then).  There
>have been plenty of groups with many Windows-based desktops available,
>sitting nearly idle 90% of the day, pretty much forever.  These systems
>have always been "free" in the sense that they are already there and
>paid for and are sitting idle.
>
>So why has development of parallel software to RUN on these "free"
>Windows systems just plain never happened?  Because the development
>PROCESS was very, very, very EXPENSIVE, that's why.  It would have been
>easier under DOS, with DOS's primitive network stack -- at least under
>DOS it was relatively easy to launch a process with its own "terminal"
>resources, so the OS would know what to do with stdin/stdout/stderr.

Very true.. however, after a decade or so of wandering in the GUIcentric 
world of the user event loop driven windows model, MS is finally getting to 
a multiprocess model that's reasonably clean.  As might be expected, it 
came first to the "C/C++" world, and most recently to the "VB" world.

>Compilers alone were (and are) expensive.

This has changed recently.  MS now gives away the basic versions of their 
development environment (they're called "express editions") for VB, VC++, 
VC#, VJ++, etc.  And even in their most rapacious, I wouldn't call MS 
development products "expensive", compared to the cost of a body to use 
them.  For years and years, the basic tool set has run around $1000-2000/yr 
for the "full-up" version, which, compared to the cost of the PC sitting on 
the developer's desk, heat, power, lights, salary, etc. isn't a huge 
amount. Yes, it's a HUGE amount compared to what the usual Linux toolchain 
costs, but, in the overall scheme of things, it's not excessive.  You want 
expensive, go price out the toolchain to develop "software" for the latest 
megagate FPGAs.  Tens of $K/yr wouldn't be out of the question.



>It was (and remains) a total
>PITA to access a node from a single seat -- Windows WANTS you do to
>everything at a system console.  Only if the result were worth a lot of
>money would it ever have been worth it, and even then it would have cost
>much LESS money under Linux (as it already did under other flavors of
>Unix) once it came into existence.

No longer.. It's not that hard to access a process on another node from a 
single seat.

Even under WinNT 4.0, it wasn't all that tedious, although there were 
inevitably hiccups.  I had a system with 4 headless machines running WinNT 
back in the 1999 time frame, where one machine was a master and fired off 
and monitored jobs running on the other machines using, almost exclusively, 
command line tools.  Much of this functionality was non-obvious, and found 
in the incredibly useful "Back Office Resource Kit", which came as part of 
your MSDN subscription (or was acquired as a freebie giveaway at various 
and sundry Windows conferences or with a variety of beta programs)


>So please understand, the cost of Windows, especially vs Linux, is NOT
>an advantage of Windows in this discussion except MAYBE in Windows-only
>shops with a high marginal cost to "start" using Linux where before they
>weren't.  Elswhere sure, MS may well give it away to try to get market
>share.  It won't matter.  They'd have to PAY people to use it instead of
>Linux to overcome the difficulties people will encounter when they try
>programming it.

I don't know that it will be all that hard to write "windows-centric" 
parallel code within the Windows Visual Studio development environment. It 
already handles multithreaded applications, and realistically, there's no 
real difference whether the thread is running on the same processor or on 
another one.  If you are used to the windows component programming model, I 
don't see writing parallel applications being a huge hurdle.

Windows is already very much  based on a "message passing" model to 
communicate among applications and between application and OS.  For years, 
all Windows programs have basically been something that receives event 
messages and generates other event messages to communicate with the GDI and 
the user. That is, I don't ever actually call a mouse driver.. I just get 
messages from the mouse like "click" or "mouseup, mousedown".  Likewise, I 
don't actually write into a video buffer, I invoke a Line method of a 
suitable object instantiated within a form object, which sends a message to 
the graphics thread to actually draw the line (or not, depending on whether 
the form is visible).

Making the leap to passing messages from one process to another to do 
computation is not a big step.

In fact, you might find that long-time Windows programmers will find MPI 
programming easier than someone who has always dealt with single threaded 
computational code.  They're used to asynchrony among the routines that 
comprise the usual application.


>  The only groups of people who will be interested (I
>think) are commercial developers seeking to make a shrink-wrapped
>product for people who want a turnkey cluster, and people who are happy
>paying out of the nose for just such a cluster.  People who are content
>with being locked into a totally non-portable schema for their future
>parallel computing needs, at that.  At the moment, at least, this is a
>fairly small chunk of all cluster usage.

For today, this is true, because cluster users, by and large, are people 
who a "researchy" and are interested in a)getting the most computation for 
the buck and b)like to tinker with the innards of the computational engine.

However, consider that there may be a HUGE market for computation that 
isn't as price sensitive, nor is interested in getting into the 
innards.  This market may, one day, wind up being vastly larger than the 
existing cluster marketplace.  For all you know, there's a burgeoning 
market of people who just want to run their giant excel spreadsheets faster.



>Also everybody needs to realize that Microsoft didn't decide "yesterday"
>that the cluster market was important.  They've been trying to crack it
>for years.  Think of the Cornell site -- a model that hasn't exactly
>proliferated, but not for lack of effort.  Think of the occasions over
>the last umpty years when a MS employee has come on list and tried to
>co-opt it and get the list to recognize Windows clusters as "beowulfs".

Sure.. and I think that they have recognized that cracking the HPC research 
community is not a productive strategy.  Much better to go after the 
"personal supercomputer appliance" market, which, in any case, is probably 
bigger, in terms of unit sales.



>Note that in any of these cases they are/will be going for that
>shrink-wrap market -- Apple and MS more competing with each other than
>with standard Linux clusters in a typical research or industrial
>setting.


I think that this is the case.  Use that cluster to run FEM faster, etc.


>Perhaps the BEST idea is to make your application (computer chess, no?)
>have a socket-based API -- maybe XML based -- and make the front end
>entirely separate.

Which, of course, is what SOAP (and the earlier XML-RPC) is all about. And 
MS is full steam behind SOAP.


>There are two, maybe three advantages that Windows continues to enjoy
>over Linux at the desktop.  One is a truly enormous desktop market share
>as a starting point,

And this makes the Windows platform, in general, an attractive one for 
would-be software developers.  There's a much wider spectrum of jobs 
available to you as a Windows developer, compared to almost anything 
else.  Sure, the average salaries may be higher for Linux or OS-X, etc., 
but face it, not everyone in the software development marketplace are high 
productivity stars like us list members.  For a huge number of IT 
professionals (as in hundreds of thousands, if not millions), you're more 
employable if you're a WindowsWeenie, especially if you want a regular old 
9-5 kind of job, slinging the code to deal with yet another health 
insurance reimbursement form.



>This includes cutting deals with hardware resellers that basically make
>it suicide not to distribute Windows exclusively and without any real
>
>As Mel Brooks once noted, "It's good to be King".  Not so good to be a
>subject...
>
>The second is device drivers and hardware devices.  Here they are
>"accidentally" aided by hardware manufacturers persistance in viewing
<snip>
>and when Linux can first use it.  Linux users have had to get used to
>the idea that they just cannot ever count on bleeding edge hardware or
>nifty electronic toys working on their systems without either waiting
>for a year or investing a lot of effort.
>
>This negatively affects the rate at which Linux has penetrated the
>desktop tremendously, perhaps more than any other thing.  It irritates
>ME and I know what I'm doing and can play the let's hack the drivers
>game in a life-threatening pinch -- if I buy (say) a brand new Toshiba
>laptop or AMD-64 box, there is a very distinct possibility that one or
<snip>
You bet... and if someone has a 16 processor cluster, and decides a couple 
years down the road to buy some new nodes, there's that whole Linux/Beowulf 
thing "Well, you should buy a few nodes, see if they'll boot, find the 
right drivers, etc." This sort of tinkering could easily take a couple 
months.  In the Windows world, you'd be pretty sure that there wouldn't be 
much agonizing.. either they boot right up, or it's a lost cause from the 
get go.


>The third is the supported application space.  For most people "Office"
>is a mission-critical application.  Forget about whether or not .doc or
>.xls formatted documents are Evil in their basic conception and design
>-- the fact is that .doc's are sent all over the place because of the
>first point above and linuxvolken need to be able to open them, read
>them, write them, mail them back -- unbroken.  Similarly, there are many
>applications that are written to "use" Explorer as a fundamental part of
>installation or operation, there are games that use Windows-based
>graphics drivers, there are applications that do "cool things" like
>letting you index your photo collection, all available for Windows
>(generally for money) but not always for Linux.

And this is the key part.  Relatively few companies are willing to pay for 
TWO different computers to sit on someone's desk (whether in terms of 
support, physical space, etc.).  In most places, you HAVE to have Windows, 
because there are mission critical applications that are native windows 
that you MUST use (e.g. filling out your timecard, doing some 
infrastructure part of your job, etc.).  So, given that you need Windows 
for this stuff, you need a HPC solution that is Windows compatible.

It is true that for some subset of the HPC needs, there are turnkey 
companies that will set up a Linux based cluster that is accessible from a 
Windows workstation, quite transparently.

>The third (application space) has made tremendous strides.  Open Office
>has all but eliminated the Office gap -- and is one of several choices
>available, as usual.  Cedega and Wine have lowered the gaming gap, with
>some users actually reporting better game performance under linux
>emulation than under native windows!  And a glance at e.g.  Fedora Core
>extras gives you an idea of what has happened to the application space
>in general -- it is literally exploding with new, cool, GUI based
>applications.  There is more stuff available in extras for linux for
>free than there is in Best Buy for Windows for several thousand dollars.

But, in large companies (the ones most likely to whip on out and buy a 
Windows Cluster), there's typically a lot of homegrown Windows software of 
one sort or another. It's bad enough trying to migrate from PowerBuilder to 
VisualStudio, but at least they're both Windows.


to be laid out and browsed in the bazaar of the possible, sampled freely
>by the end user, all without spending a penny.

Except for all those mission critical "timecard fillout","vacation 
request","service order to get the lights in my office fixed" kinds of 
infrastructure stuff, not to mention all the software that might have been 
developed to meet the business's core objectives.  If you've got 20,000 
call center operators all toiling away on your custom Windows application, 
developed/evolved over many years at a cost of millions of dollars, as they 
deal with the general public, you're not going to be too impressed by the 
Linux claim of "free development tools" and "free interesting 
widgets".  It's a tiny, tiny fraction of the cost of the development costs 
of the product.



>I honestly think that the desktop software gap is pretty much closed,
>and is if anything leaning inexorably over towards linux.  After all,
>once a really great GPL application is released for Linux, it tends to
>stay "forever" and only improve.  There are only so many applications
>most people are likely to use or need.

I'm not sure I agree with this assertion.  It's certainly true in 
mainstream "typewriter" kinds of applications, but that's kind of like 
saying that any telephone will work plugged into the analog jack, and you 
don't need one made by Western Electric.  Everyone needs a phone on their 
desk, but, typically, they also need something that is company specific.

Oddly, in the home market, I think your assertion IS closer to the 
truth.  In the consumer area there is a reduced diversity of needs (or, at 
least needs that can be met by inexpensive consumer software.. nobody is 
going to plan their daughter's birthday party using Primavera or use SAP 
for their home budgeting)

>   When EACH person's application
>space is covered, the marginal cost of the linux-windows move (in either
>direction) is dominated by the REAL cost of Windows vs Linux per se, a
>price war Windows can literally never win at least once the hardware
>device issue is resolved.  Numerous surveys have shown that the number
>of Linux developers continues to rise and overtake the number of Windows
>developers,

I'd be interested in the source of that bit of information. I have a hard 
time believing that there are, in an absolute sense, more skilled Linux 
programmers than skilled Windows programmers, particularly in the  middle 
tiers (say, from 5-10 years experience).  If you survey people in IT, yes, 
a lot of them will say that they've used Linux and programmed for it, but 
that doesn't necessarily mean that they are ready to go out and do 
production coding.  In the academic world, always a bastion of *nix, I'd 
believe it (and in fact, I'd also believe that skilled WindowsWeenies 
happening to wind up in the halls of academe would probably hide their 
light under a bushel, for fear of being tagged as an agent of satan).



>At this point one place where the software gap persists (and is VERY
>DESTRUCTIVE to Linus's plan of dominating the universe) is in business
>middleware.  This is the one place on the planet where people want,
>need, absolutely insist on shrink-wrapped solutions, and Linux has not
>proven itself capable of filling that need with shrink wrapped software.

Especially, I would think, "internally developed" shrink wrap software. My 
wife used to manage a crew of around 100 developers whose whole purpose was 
to develop business middleware specific to the business.  From an internal 
standpoint, it would be rolled out as a standard product to all the 
thousands of users within the business, no different than a release of Word.


>There are solutions, sure, but they tend to be GPL projects, underfunded
>and understaffed.

And totally unusable for any practical business. Most middleware has a 
goodly amount of customization inherent in it.  Consider something like a 
Human Resources package for a company with several hundred people.  Pretty 
much every company has the same generic functional needs (keeping track of 
benefits, hiring, firing, etc.), but everybody does it slightly 
differently.  So, the middleware vendors include the customization in the 
(relatively high) selling price.  The business gets a fairly reliable, but 
sophisticated piece of software. The vendor has a fairly componentized and 
modifiable code base to start with, so the non-recurring cost for each 
customer specific version  is reasonable.  Furthermore, for each customer 
requesting a new function, there's a possibility that the function might 
wind up being in the codebase for the "standard product".

The same sort of thing applies to accounting.  Everyone follows the same 
rules, pretty much, but everybody has a different chart of accounts, 
different other packages that the accounting system has interact with, etc.



>  Hobbyware, as it were.  It just isn't "fun" to build,
>design, maintain business middleware, and nobody has realized that it is
>perfectly possible to build AND SELL commercial software in this rather
>huge market without much risk that OS developers will come along and eat
>your lunch anytime soon.  Indeed, what MOST companies want to buy here
>is a direct support line and confidence as much as software anyway.  I
>personally think it is a tremendous business opportunity waiting for
>somebody to realize it and sell turnkey Linux-based business middleware
>that can talk with equal ease to clients on Lin or Win desktops.
>Integrated accounting, payroll, POS, inventory, HR -- dull as molasses
>to code and maintain, but absolutely essential to a myriad of small to
>midsize businesses that OTHERWISE have a strong interest in running
>Linux top to bottom to minimize the overall costs of IT.

exactly

>Ultimately, I agree with the earlier assessment (from Jim?), that the
>entire HPC market is no more than a pimple on MS's behind as far as
>likely contribution to MS's bottom line is concerned, and suspect an
>ulterior motive in their entering the market -- like trying to at long
>last put a finger in a gradually widening crack in the leaky dike that
>supports their product all nice and dry and monopolistic and dominant
>inside its protected ring of FUD.



Jim