[Beowulf] MS HPC... Oh dear...

[Jim Lux]

At 07:30 AM 6/12/2006, Robert G. Brown wrote:
>On Mon, 12 Jun 2006, Geoff Jacobs wrote:
>
>Sigh.
>
>The issue here is strictly one of commercial software.  I very much
>doubt that you'll see many folks who roll their own parallel software
>migrating in droves to MS clusters for precisely these reasons -- they
>have to write their own code, debug their own code, and haven't the
>time, the staff, the budget to deal with cryptic error codes and
>commercial support mechanisms with built in obfuscation and delay.


precisely.  This is aimed at people who, for whatever reason, want to be in 
the Windows environment.


>So will they come?  Companies that are trying to SELL ready-to-run
>parallel applications will simply love this.  It gives them a definitive
>target platform to code for (unlike the plethora of rapidly evolving
>linux based distros and their associated clusters).

You bet.

   Only people who really don't
>give a rodent's furry behind about money will be willing to spend a ton
>of money for a product where it takes days for your cluster-ignorant
>MCSEs just to figure out the licensing arrangements for the nodes (and
>to learn what a node IS).


It's probably not that bad, most MS customers (at least big ones) are very 
cost/benefit oriented. However, if the MS Windows solution allows one 
$200K/yr (burdened) engineer to be twice as productive, the solution can be 
pretty expensive, and still be a net "good thing" for the organization 
forking out the bucks.  A lot of places might not look beyond the "it's 
better than nothing" approach rather than "is it the best of all possible 
alternatives".



>So IMO the MS move is targeted at a very carefully specified market,
>with little or no interest or hope of expanding beyond that market.
>Their potential customers are those who will build large to very large
>clusters to run very specific, commercially sold and supported software,
>and will have very deep money-is-no-object-because-it-is-not-our-own
>pockets.

Exactly.  You're selling a "capability", not a "platform for future 
use".  I don't see many "end users" buying Windows Clustering Edition (or 
whatever it's called).  I see it being part of a (much) bigger 
package.  Here's your Gene Sequencing Box for $2M..




>  NIH comes immediately to mind, as does Hollywood (less so).
>These are folks that are happy to spend money like water as long as
>things "just work" according to their particular definition of the term,
>which generally means "so that our team of six FTE MCSE admins can keep
>it running transparent to us".  They won't even care if things work
>OPTIMALLY and many will not even understand how to assess things like
>parallel scaling -- they just want them to work, and probably to dump
>their results directly into Office Pro so they can make their power
>point presentations with nifty excel figures without any intermediate
>data import step or the hassle of managing two operating systems.


Exactly.



>Will Microsoft's move succeed?  Probably.  If and only if they get
>ENOUGH commercial software ready to plug and play AND somebody willing
>to buy it, although they could lose money on this for years and still
>fund it just to be able to claim market presence and not think twice
>about it.

FWIW, Mathworks has announced their Windows Cluster compatible version of 
Matlab.

-- its biggest advantage is in visual/windowed stuff, e.g. VB.  What
>good is VB in a grid, where your application CANNOT do anything
>whatsoever with a GUI?

I can see someone with an application that does some serious number 
crunching, currently implemented in Visual whatever with multiple threads, 
being ported to the cluster edition, where each thread runs in it's own node.

There are a surprising number of applications being written in a front 
end/backend sort of paradigm, where you have a thread running the UI and 
one or more worker threads running the computations.  Granted many, many of 
these are more classic client/server type applications.

However, it may well be that there are partitionable problems that are not 
just like running a server farm (the latter which is really a EP sort of 
application).




>So, naaaaa, not likely to be a popular development platform for real
>researcher's writing their own code or using open source code.
>Commercial only.
>
>So the real (rhetorical:-) question is:  Who is writing the commercial
>software that will run on this? and: Is anyone out there going to buy it
>at rates that scale per node, given that one is basically SPENDING nodes
>to get it?  It isn't about the OS, it's about applications,
>applications, applications.  One has been able to develop and run
>parallel applications on WinXX systems for a long time now, really --
>pretty much as long as on linux if not longer.  PVM for WinXX existed
>back when Linux was but a gleam in Linus's eye.  There is a REASON that
>even way back then, when WinXX was much CHEAPER than commercial Unices,
>it was a most unpopular platform for parallel code development, and that
>reason hasn't really changed.

I would say that in many ways, MS has taken the just criticisms of earlier 
versions of windows to heart, and newer versions are much more close to the 
*nix process/task/data model than one might think.

All Windows prior to NT were basically GUIs on a single threaded DOS 
underpinning.  However, since NT (including 2000 and XP and Vista) the OS 
is a "real" multitasking/multiprocessing kernel with all the usual bells 
and whistles.  And now, the development languages and tools are catching 
up.  Up until, say, Visual Studio 2003, most of the development languages 
showed a strong single threaded/DOS-ish heritage (heck, most of them could 
generate DOS executables).



>"cluster" on top of ANY operating system that groks "network".  The
>clustering team is probably tiny and cheap, as MS project typically go,
>and at the moment is probably spending more time ensuring that there are
>commercial apps ready to go than they are ensuring that there is
>anything particularly "cool" about the clustering environment itself.

I would assume that this is the case.


James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875