[Beowulf] MS HPC... Oh dear...

[Vincent Diepeveen]

> "Robert G. Brown" <rgb at phy.duke.edu>
>> On Mon, 12 Jun 2006, Vincent Diepeveen :
>> We will confront you with your statement in a few years from now.
> Go for it;-)

> Note to all others:  The following is a patented rgb rant with no
> otherwise meaningful content.  Hey, it has been a while...:-)

Oh dear now the back of that beer mat will get REALLY filled up.

> Anyway, feel free to hit the "d" key now and skip it.

>> If microsoft doesn't price their server/cluster stuff too expensive then 
>> in X years from
>> now they'll dominate the highend market. Microsoft always has just taken 
>> markets by
>> storming in giving away copies of their software for near free initially.
>>
>> Competition is hardly possible against that from software viewpoint.
>
> Um, Vincent, you ARE aware that Linux is FREE software -- as in FREE
> free, free as in air, beer, birds -- aren't you?  As in I haven't paid a
> company for linux in years and years, and only bought the occasional
> box-set copy of Red Hat back when I did as a voluntary contribution?
> To turn your own observation around, "Competition is hardly possible
> against that from software viewpoint"... especially since Linux "HPC"
> isn't being given away free "initially" -- it is guaranteed to be free
> free by the GPL and other open licenses used throughout pretty much
> forever.  This isn't a case where Microsoft can, um, "undersell" its
> competitors short of giving them money along with their product -- which
> I fully expect them to do, by the way, especially initially -- and any
> effort they expend in this direction is just money pissed away except in
> very limited and specialized commercial markets.

You and i are like junkies busy downloading all kind of linux distributions 
'for free',
sitting and waiting a LONG time (in your case expensive time paid by the 
government,
in my case i just CLAIM my time is expensive as i'm running my own company).

Then we burn some DVD's and CD's in order to find out after a few weeks of
trying that Ubuntu nowadays is the best distribution. The rest is utter crap 
(from the
FREE distributions) now that we have 64 bits dual core monster chips.

OTOH sure microsoft undersells in practice. The average Joe walks to his or 
her neighbour
and asks for a free cd recordable from his neighbour with 'windoze' at it.

Directly without lifting a single eye brow, the neighbour hands through the 
door a cdrom.

Dang we've been beaten in price BIGTIME then already, not to mention TIME.

> You also missed another of my points.  It has been possible to write
> parallel software that runs on Windows boxes since maybe 1993 (can't
> recall the exact date that somebody did the Windows port of PVM, but I
> vaguely recall seeing Windows ifdefs in the source about then).  There
> have been plenty of groups with many Windows-based desktops available,
> sitting nearly idle 90% of the day, pretty much forever.  These systems
> have always been "free" in the sense that they are already there and
> paid for and are sitting idle.

I remember having a discussion a year or 13 ago with a professor who said 
that his university
always would have sun or hp boxes. His systemadmin wasn't happy with that 
statement and
semi publicly complained about it. He just saw that for 130 students there 
was just 20 sun boxes.

With PC's it would've been possible to put 60+ boxes there at the time, 
besides that they were
faster also, about 3 times faster objectively. On paper of course Sun and HP 
had great whitepapers
supporting a good speed and a low price for those boxes for each thing 
achieved.

Standardizing for the masses *always* wins in price from all those small 
dedicated solutions
that eat money and extra service personnel as the hardware is too complex to 
understand
for the average joe.

> So why has development of parallel software to RUN on these "free"
> Windows systems just plain never happened?  Because the development
> PROCESS was very, very, very EXPENSIVE, that's why.  It would have been
> easier under DOS, with DOS's primitive network stack -- at least under
> DOS it was relatively easy to launch a process with its own "terminal"
> resources, so the OS would know what to do with stdin/stdout/stderr.
> Compilers alone were (and are) expensive.  It was (and remains) a total
> PITA to access a node from a single seat -- Windows WANTS you do to
> everything at a system console.  Only if the result were worth a lot of
> money would it ever have been worth it, and even then it would have cost
> much LESS money under Linux (as it already did under other flavors of
> Unix) once it came into existence.

> So please understand, the cost of Windows, especially vs Linux, is NOT
> an advantage of Windows in this discussion except MAYBE in Windows-only
> shops with a high marginal cost to "start" using Linux where before they
> weren't.  Elswhere sure, MS may well give it away to try to get market
> share.  It won't matter.  They'd have to PAY people to use it instead of
> Linux to overcome the difficulties people will encounter when they try
> programming it.  The only groups of people who will be interested (I
> think) are commercial developers seeking to make a shrink-wrapped
> product for people who want a turnkey cluster, and people who are happy
> paying out of the nose for just such a cluster.  People who are content
> with being locked into a totally non-portable schema for their future
> parallel computing needs, at that.  At the moment, at least, this is a
> fairly small chunk of all cluster usage.

> Also everybody needs to realize that Microsoft didn't decide "yesterday"
> that the cluster market was important.  They've been trying to crack it
> for years.  Think of the Cornell site -- a model that hasn't exactly
> proliferated, but not for lack of effort.  Think of the occasions over
> the last umpty years when a MS employee has come on list and tried to
> co-opt it and get the list to recognize Windows clusters as "beowulfs".

You are giving yourself the reason why windows will take over the highend in 
terms of software.

Development time and costs, especially from drivers.

If 95% of the users from Myri in the far future just wants a windows driver 
that's fast for their $150 highend network
card with mass produced standard fiber cables in India from $20 a piece,
i'll have to see that there will be a linux driver for it getting developed.

If they can earn 20 times more on the windows users,
they could have next reasons why they don't want to develop a new fast linux 
driver for it:

a) they earn less on those 5% of the users as the cost for every flavour of 
the kernel they need to
develop a new driver for. So the break even point is at > 5%. In reality the 
break even point
will of course be X times more than the % of cost. About factor 5 to 10.

So for example if for 1% of cost they can make 500% profit on that 
investment, they might consider it.

b) microsoft demands that they need a time advantage for such a driver. no 
linux driver may be released
until 2 year after product gets sold onto the market with windows driver.

Let's say it very polite: Option B is not uncommon in this world.

> Heck, back in the days of NT they offered to give >>me<< NT licenses for
> whole clusters of Dell computers we'd gotten as part of an Intel
> equipment grant and assign us our very own Microsoft-paid software
> engineer to be our very own slavey to facilitate the porting of code and
> all if only we'd consider running Windows on our clusters instead of
> Linux (where we had to WORK to get linux to run at all, mind you).  They
> made noises about giving us access to OS source code and everything.  We
> wanted to get work done instead, and declined.  The SMP systems, running
> linux (mostly 2.0.x!) throughout, were finally retired years later with
> a record of maintaining a duty cycle in the high 9's over their entire
> lifetime -- basically never crashing except due to hardware failure,
> once we got their adaptec drivers stabilized.
>
> It is also well worth remembering that in larger institutions, running
> linux servers is ALREADY well-known (and has been so known for years) to
> be cost effective relative to WinXX servers for so many, many reasons.
> In fact, a lot of places run linux servers and e.g. samba to service
> their windows clients.  Look at the cost scaling of Windows server
> licenses to Windows clients some time -- the number of clients they say
> you can support before you need to buy another server.  A single Linux
> server can handle many times more Windows clients than a Windows server
> -- for free.  Look at security.  Look at ease of maintenance, especially
> remote access and maintenance.
>
> I honestly don't think they're going to find a lot of people who go
> "Gee, at LAST, now we can do cluster computing and not have to support
> linux any more". It will be more an issue of either "Gee, we already run

Back in 1900 there weren't many cars on the road.

If you were talented with bicycles then you could try make a car your own 
and sell it.
Some did.

Nowadays cars are very advanced.

They have to be mass produced, or they get expensive.

The tiny dutch Spyker isn't really massproducing cars. They're 600k euro a 
piece
as a result of that. Every Arab sheikh who selfrespects his billions owns a 
Spyker.

Collectors are there to stay.

It's a matter of time before windows will work fine and stable at big 
supercomputers,
just like it works pretty ok at quads and duals now.

If it does and all those cluster manufacturers give you 2 options,
that's buy a cluster with windows, or buy a cluster
without windows for the same price.

> linux servers and clusters, why in the hell would we use this unless you
> PAY us to port to it and use it (which is pretty much what they did at

Pay you to port something?

You see it wrong.

You can get system time at a windows supercomputer for free from
your university. You can either take care that you sit forever in that
commission that decides which supercomputer you go buy now,
or you can port your own program to windows.

> Cornell)" or "Gee, we're already paying a ton of money for our twenty
> nfive copies of Windows Server to run our 125 clients, we'd simply LOVE
> to pay you two tons more to get 1024 Windows Cluster licenses running
> from umpty head nodes, as long as you don't make us learn that nasty old
> Linux..."  The latter argument being put forth, of course, by the
> well-entrenched admin staff consisting of MCSEs, just as once upon a
> time not so long ago it was put forth by IBM mainframers and COBOL
> programmers and DECnet administrators and...

> So sure, Microsoft will doubtless define whatever they accomplish here
> as "success".  They've been trying to crack the cluster marketplace for
> something like eight or nine years now, at least -- they started as soon
> as the Top 500 started to be dominated by cluster after cluster, none of
> them using Microsoft products of any sort on them.  Without success --

Let's hope it'll take them real long to crack it and let's hope that
manufacturers will all of them not let themselves get blackmailed
by microsoft...

But let's face it, until recently open-gl looked great and portable to other 
OS-es.

The reality is that in future we will have to rewrite to directx.

Just google on what happened to open-gl 2.0 standards and why
microsoft won those courtcases.

> the cluster market has not been terribly tolerant of cost-inefficiency

Yes the supercomputer market is pretty cost inefficient.

If it stays like that, then you can vote in that commission for an expensive
linux supercomputer, if it doesn't then obviously the doom scenario i
wrote on the upside of the beer mat is gonna happen.

> and indeed is one of the most visciously cut-throat marketplaces on the
> planet in many ways, and Windows makes Microsoft a huge profit margin
> for a REASON, and that reason ain't its end-user cost efficiency... or
> its high quality and features.  It's because they achieved a monopoly
> the old-fashioned way -- by driving its relatively few competitors out
> of business while biting the very hand that made them what they were at
> the time (IBM's).  Mostly.  Enough.  Even then they only succeeded
> because Sun Microsystems was stupid and didn't drop the cost of their
> perfectly usable x86 Unix to $50/seat on Intel hardware, bribe into
> existence some mission critical software, and get there fustest.

How many compilers are there left for windows?

Visual c++ is there. Which others?

There isn't a single serious compiler for windows left other than
visual c++.

Why?

> So, perhaps they've finally identified that ideal rich-but-stupid
> segment of potential cluster customers that can make them high-margin
> money; perhaps they've decided that they have to get into the market
> even if it is a dead loss forever or lose market share elsewhere, who
> knows?  We'll see how long it takes for them to buy themselves a top 10
> cluster somewhere, like Apple did a few years back.  Did the apple
> cluster materially affect the dominance of linux/x86?  It did not.  Wil

You should really start worrying here.

Microsoft in past also copied Apple and really grew big with those
stolen ideas.
l
> Microsoft's playing exactly the same game make any significant
> difference?  I honestly doubt it.  And who is going to help them?  IBM

Wintel?

For some years i've had this nightmare already of a hardware chip
that just works for windows.

Just because intel is making quite some billions a year,
m$ can't really make a clear appointment there i bet.

How many companies in highend make several billions a year?
Sun, IBM...

At least one of all those companies in highend will grow real big by
making an exclusive deal with microsoft.

So as long as you are willing to buy an IBM or Sun platform,
you might be able to keep windows outdoors.

I do believe however in a wave effect. Sometimes a company makes
a good product, then next year some other manufacturer has a better one.
And a few years later your wave peeks again and you have the best
product again.

If that's the case then you'll have to buy sometimes really shitty hardware
to keep linux.

> still smarts from being screwed over OS/2 and is just itching to get
> oh-so-polite revenge.  They pay lip service to Microsoft where
> necessary, but inside IBM they tend to LIKE linux.  Apple may run
> Office, but they despise Microsoft.  Hardware vendors may well be
> arm-twisted into fronting them on HPC as they have desktop Windows in
> the past, but only if there is a huge market demand, not the other way
> around (to create such a demand).
>
> Note that in any of these cases they are/will be going for that
> shrink-wrap market -- Apple and MS more competing with each other than
> with standard Linux clusters in a typical research or industrial
> setting.
>
>>> From my multiprocessor product i'm not releasing a linux multiprocessor
>> version, to
>> give 1 obvious example.
>>
>> Porting the GUI to linux is simply too much work, even though we are 
>> open-gl at the
>> moment and porting should be theoretically possible with just X weeks 
>> work.
>>
>> Microsoft dominates because all GUI's are running under windows in a way
>> that users can work with it.
>
> That's your choice.  It's probably a wise one -- all users can work with
> a GUI on ANY system (that's the whole idea:-), but Linux users are
> notoriously unwilling to pay people for software in the first place.

It's not a matter of you not being prepared to pay only.
There is no 'linux only supermarkets'. (where supermarket = big 
distributors)
Supermarkets are willing to put certain windows products there.
*that's* why windows software sells.
BECAUSE it is in the supermarket.

If supermarkets would accept linux products, i would of course
hand them over a linux product.

GCC is for free, and open-gl also works fine at linux.

So i can cut costs to produce a linux version.

But who is gonna sell it for me?

Products sell because users see the product.
If microsoft is going to push a cluster windows version,
then i'm basically busy with: "how can i produce a product
that works great for cluster windows, because tens of millions
will want to get that cluster windows, all i need to do is sell
them an app they like to toy with".

> They'd be more likely to look at what you've got and clone it.  However,

Ah i knew you already knew that Microsoft and Apple story :)

> I do think that true wisdom is writing a GUI that is cross-platform
> portable by design, and not locking yourself into or out of any
> particular market, if possible.  If you're using Open GL it sounds like
> it should be possible.  Dunno.  I personally am fond enough of Glade and
> Gtk, which is purported to go the other way, but my needs are simple.
> Yours may not be.

Not easy to be cross platform.

Users want so many features that work great in windows and easy
and not so great under linux.

The price of being cross platform is that your product no longer gets
sold in a supermarket.

To mention one simple problem under linux; fonts picked by designers
that were expensive paid and look great in windows, if you display that
same thing in linux, somehow linux messes it up there. The font is of
other size and looks different.

So most windows needs redesign when simplistically recompiling
a simple win32 API.

Or in fact, just running it in wine :)

> Perhaps the BEST idea is to make your application (computer chess, no?)
> have a socket-based API -- maybe XML based -- and make the front end
> entirely separate.  That way people would probably write your game
> interface for you -- right into the existing Gnuchess program, most

Yeah masses line up.

Everyone works for free for you.

> likely -- or build it as a PHP or java or Gtk app on top of your API.
> This really divorces the choice of end-user platform from the actual
> compute engine/cluster, and lets you focus energy where it makes sense,
> probably in the latter once you have ANY sort of simple GUI running that
> can talk to the engine.

JAVA was a nice try from Sun to accomplish exactly that.

JAVA is too slow for games however.

We are busy with realtime software.
Speed matters.

The videocards are so slow anyway of the average pc, that we'll need quite a 
bit of
system time to render 3d scenes.

>> That said, their huge advantage is dissappearing a bit, as lately i'm 
>> under the impression they
>> no longer have the best programmers onboard now that their stock/shares 
>> don't yearly double.
>
> There are two, maybe three advantages that Windows continues to enjoy
> over Linux at the desktop.  One is a truly enormous desktop market share
> as a starting point, coupled with a robber-baron mentality in the
> software marketplace in general that would have made Cornelius
> Vanderbilt blush with shame -- or regret that he didn't think of it
> first.  For many, many years, MS has shot down any possible threat and
> then clubbed the corpse until it stopped twitching, "friend" or foe
> alike, whereever the law and billions of dollars in high-margin net
> profit permitted.  With devastating effect.  Where is Borland today?
> Lotus?  Corel?  Netscape?  IBM and OS/2?  And the list goes on, and many
> of the products that remain that don't actually SAY Microsoft on the
> cover are sold by companies that MS owns a chunk -- sometimes
> controlling chunk -- of (as was recently observed on list regarding
> rendering software).  It's an utterly old-fashioned monopoly.

Let's do you a public offer.

If you donate me 10000 dollar i'll port our 3d GUI also to linux
and start selling it for the same price like the windows version.

The port will cost me more in fact as support for each version will
eat way more than $10k in human resources, not to mention the
designers and programming time it takes us (we are with 2 programmers
fulltime and 1 parttime). I'm willing to take *that* risk.

The first supermarket boardplaying 3d product for linux.

Think about it. You could make history here.

Feel free to do some postings online. I'm sure you'll find a 100 users
willing to donate you $100 money to have that port done.

I bet 90% of those users doesn't even know how to manage
to put the shared memory bigger in linux (you need root access
to set it from 32MB to something larger and then you have to
explain to them what root access is). Just that thing already
will cost me $10k to support i bet, emailing the faq having
that info...

to set it to 2 GB as root:
  echo 2000000000 > /proc/sys/kernel/shmmax

> This includes cutting deals with hardware resellers that basically make
> it suicide not to distribute Windows exclusively and without any real
> user choice in the matter (as in "no operating system installed" not
> even appearing as a menu option or consumer choice, let alone next to
> "install Fedora Core 5" for $25).  Buying a controlling interest in any
> company that has a popular linux version of its software -- and putting
> an end to the linux version on the spot.  Helping companies do the
> required integration engineering to ensure that their product (hard or
> soft) runs under Windows out of the box, and ideally on nothing else
> ever.  Cloning any really valuable software tool, rebranding it, playing
> games with the OS interface that create a perception among customers
> that the competing products are unstable, and sowing FUD until they have
> a comfortable 70% of the market or so.

> As Mel Brooks once noted, "It's good to be King".  Not so good to be a
> subject...

> The second is device drivers and hardware devices.  Here they are
> "accidentally" aided by hardware manufacturers persistance in viewing
> device drivers for their own devices as some sort of IP, and resistance
> to the very concept of the open ABI as anathema, lest they make it even
> easier for their product to be cloned by Taiwanese silicon foundries and
> released for 1/3 the cost.  Which they inevitably are anyway.

> Linus, on the other hand, has absolutely insisted that the linux kernel
> will not EVER be made friendly and tolerant of binary insertions.  The
> combination creates a very definite, very annoying lag between when
> hardware first appears (supported by Windows out of the box, of course)
> and when Linux can first use it.  Linux users have had to get used to
> the idea that they just cannot ever count on bleeding edge hardware or
> nifty electronic toys working on their systems without either waiting
> for a year or investing a lot of effort.

> This negatively affects the rate at which Linux has penetrated the
> desktop tremendously, perhaps more than any other thing.  It irritates
> ME and I know what I'm doing and can play the let's hack the drivers
> game in a life-threatening pinch -- if I buy (say) a brand new Toshiba
> laptop or AMD-64 box, there is a very distinct possibility that one or
> more of its components or even the motherboard's basic chipset will not
> be recognized by Linux (been hit by both recently) at least not unless
> I'm using a bleeding edge version of the kernel or a distro and do a
> fair bit of googling and maybe some development version building
> (something not everybody can do).  It's one of the things that makes
> e.g. NDIS so very interesting -- if Linux ever DOES get to where a
> "universal device driver" functions that can use any given native
> Windows binary driver -- even at some small cost in efficiency -- it's
> going to remove one of Windows' most persistent advantages in the
> desktop market.

It wouldn't be the first time when the monopoly of microsoft
breaks another good initiative.

Drivers is the most important thing there is.
Without good drivers open-gl nor directx will be fast.

Making users as easy as possible to install drivers is very important.
the default windows drivers are slow for 3d.

Yet nowadays many know how to fix that problem.

A lot of people criticize windows always for its easy acceptance of
different DLL's.

No matter how buggy that system is, it's always still better than the
*.so of linux.

With linux you have the GARANTUEE that if you install something new
that the *.so system will mess up. In windows at least you've got a fair
chance that it doesn't break down as they tested it real well.

If linux would have as many big apps with their own *.so libraries like
windows has, then in general people would not be so happy about
linux and drivers/library support.

Both windows and linux have done this in a crappy way.

Yet the windows way to patch things is far more effective from user
viewpoint than what linux is doing there.

Not because windows concept is better in general than the linux .so
one. In principle it sucks just as much, if not more. However...

A huge advantage windows has in general over everything else
is that they really test things very well at windows headquarters.

Really a LOT of bugs get removed then.

In Linux that simply isn't the case.

The word 'testing' is limited to the developer trying it at his own machine
for 2 minutes, if any of such action happens anyway.

If you ask for bugfixes the standard answer is: "it worked fine here".

Then the second email shipped back is: "you know, i do this in my sparetime"

After a very extensive bugreport, describing deterministic every action 
taken and offering to
login here to see the problem themselves, then sometimes a week later the 
third email is:
"i found a bug" and sometimes the
third email is: " F' off, patch it yourself".

Now that my political skills seem to improve with linux folks,
the second version of the third email tends to get more seldom.

Freeware Filezilla windows server is another recent successtory of that 
kind.

> The third is the supported application space.  For most people "Office"
> is a mission-critical application.  Forget about whether or not .doc or
> .xls formatted documents are Evil in their basic conception and design
> -- the fact is that .doc's are sent all over the place because of the
> first point above and linuxvolken need to be able to open them, read
> them, write them, mail them back -- unbroken.  Similarly, there are many
> applications that are written to "use" Explorer as a fundamental part of
> installation or operation, there are games that use Windows-based
> graphics drivers, there are applications that do "cool things" like
> letting you index your photo collection, all available for Windows
> (generally for money) but not always for Linux.

> Of these three, the first continues to erode.  Slowly, to be sure, but
> surely (note that I'm talking strictly about the desktop, as MS's server
> share has gone up in recent years at the expense of other Unices, at
> about half the rate of linux's server share increase).  Note also that
> there is lots of evidence that the market share published for MS vs
> Linux in BOTH the desktop and the server market is signficantly inflated
> by the sampling methods used -- it typically counts all the systems in
> the world sold with Windows pre-installed, for example, and fails to
> count most of the copies of absolutely free linux that are installed
> right in on top of those same systems.  It tends to count COMMERCIAL
> linux sales, that is, which of course ignores the fact that most linux
> use by far is by people who do not pay for it.  Do they count all the
> systems installed from the mirror servers at Duke, for example?  I don't
> think so.  How could they?

Now that wintel might get back with a good system you can assume that
you wish the above numbers would have been reality in 2006.

If wintel has a new good highend system then windows in general
can be expected to sell real well highend software.

> This would be thousands of systems ON campus, all "invisible" to current
> surveying techniques, and tens or even hundreds of thousands of systems
> off campus, and Duke is just a single primary mirror out of many, and
> then there are secondary mirrors, tertiary mirrors....

> Not even repo server logs can help you figure out just how many -- the
> software is distributed directly from online repos in a mirror TREE, and
> nodes branch out to actual systems at all sorts of levels in the tree.
> As in I have around ten linux systems in my HOUSE and a complete mirror
> of both FC4 and FC5 (x64 and i386 both) to support them from a single
> rsync of a mirror of a mirror of the FC toplevel repo, and I live
> relatively close to the TOP of the tree as Duke has a toplevel mirror.
> Then there is the rest of the world, where your choice is pretty much to
> steal Windows (commonly enough done, sure, even in the US or Europe) or
> get Linux legally, for free.  Only one comes with a huge base of free
> support, with compilers, with web servers, with the ability to run
> client/server networks securely.  I wouldn't be surprised if the
> worldwide Linux "market" share (measured in installed linux desktops vs
> installed WinXX and Mac desktops) is three or four times what e.g.  IDC
> acknowledges, and it is rapidly growing as linux distros come into
> existence that FOCUS on the desktop and appear to be very popular.

> The third (application space) has made tremendous strides.  Open Office
> has all but eliminated the Office gap -- and is one of several choices
> available, as usual.  Cedega and Wine have lowered the gaming gap, with
> some users actually reporting better game performance under linux
> emulation than under native windows!  And a glance at e.g.  Fedora Core
> extras gives you an idea of what has happened to the application space
> in general -- it is literally exploding with new, cool, GUI based
> applications.  There is more stuff available in extras for linux for
> free than there is in Best Buy for Windows for several thousand dollars.
>
> With yumex one can now SHOP the linux repo chain for those applications
> as never before.  Software that either doesn't exist period for WinXX or
> that exists but costs hundreds of dollars for WinXX is a few mouse
> clicks and short download away.  Yum may end up being the ultimate
> "Windows Killer" application -- in addition to fully automating software
> maintenance and providing security updates literally overnight (there
> have been linux exploits where the gap from publication to automated
> installation of patched updates EVERYWHERE IN THE TREE is as little as
> 24 hours -- not a lot of room for crackers to get traction in there, is
> there:-) yum now permits a truly vast range of available linux software
> to be laid out and browsed in the bazaar of the possible, sampled freely
> by the end user, all without spending a penny.
>
> I honestly think that the desktop software gap is pretty much closed,
> and is if anything leaning inexorably over towards linux.  After all,
> once a really great GPL application is released for Linux, it tends to
> stay "forever" and only improve.  There are only so many applications
> most people are likely to use or need.  When EACH person's application
> space is covered, the marginal cost of the linux-windows move (in either
> direction) is dominated by the REAL cost of Windows vs Linux per se, a
> price war Windows can literally never win at least once the hardware
> device issue is resolved.  Numerous surveys have shown that the number
> of Linux developers continues to rise and overtake the number of Windows
> developers, something that is of course really hard to explain if the
> surveys concerning "market share" or the perception of real computer
> people were anywhere NEAR correct.  Developers are voting with their
> feet, or in this case their fingers.

b.t.w. I'm slowly starting to suspect you're keyboard speed must be similar 
to mine.

> At this point one place where the software gap persists (and is VERY
> DESTRUCTIVE to Linus's plan of dominating the universe) is in business
> middleware.  This is the one place on the planet where people want,
> need, absolutely insist on shrink-wrapped solutions, and Linux has not
> proven itself capable of filling that need with shrink wrapped software.

Well i have some important statistics for you.

The vaste majority of companies in EU is 1-7 persons.

In fact roughly 95% looks like that.

> There are solutions, sure, but they tend to be GPL projects, underfunded
> and understaffed.  Hobbyware, as it were.  It just isn't "fun" to build,

I've tried to support several open source projects from single persons
by simply shipping testreports clearly indicating bugs. Donated source code
to GPL projects.

Let's hope others do that too.

Yet the fact that filezilla ftp client is a great ftp client doesn't mean 
that
all people use it. Majority simply pays MONEY to buy a ftp client that
sucks a lot.

There is plenty of examples there.

> design, maintain business middleware, and nobody has realized that it is
> perfectly possible to build AND SELL commercial software in this rather
> huge market without much risk that OS developers will come along and eat
> your lunch anytime soon.  Indeed, what MOST companies want to buy here
> is a direct support line and confidence as much as software anyway.  I
> personally think it is a tremendous business opportunity waiting for
> somebody to realize it and sell turnkey Linux-based business middleware
> that can talk with equal ease to clients on Lin or Win desktops.
> Integrated accounting, payroll, POS, inventory, HR -- dull as molasses
> to code and maintain, but absolutely essential to a myriad of small to
> midsize businesses that OTHERWISE have a strong interest in running
> Linux top to bottom to minimize the overall costs of IT.

It's all about marketing IMHO. Users buy what marketing tells them to buy.
You can educate users to believe whatever nonsense if you market it at the
right way.

The more fantastic the story looks like the more they believe it.

Only after that marketing has been done properly and earned you cash
then we both agree that the support for the product is the important thing
to do right. That can make or break the product after it has been sold.

> The hardware/device problem persists, alas.  Linux printer support and
> graphics device support have improved tremendously (and the associated
> time gap has shrunk accordingly) as a number of linux vendors have
> correctly realized that this is really the last place where Windows
> holds a significant advantage and focus resources here.  Network support
> is also improving tremendously, with chipset initiatives, vendor
> support, and NDIS promising to close the time gap while a native driver
> is developed and handle edge cases.  NetworkManager proves to make linux
> networking at long last user friendly and automagical, even in complex
> environments.  USB devices have fortunately tended to be standards
> compliant, and Microsoft hasn't managed to monkey with the standards in
> such a way that gives them a meaningful advantage here, and USB support
> is now pretty good and even automagical.  Multimedia (CD's, DVD's) tend
> to just work, although DVD playing (for example) tends to be suboptimal
> unless you get the driver thing perfectly worked out for your hardware.

Well i never managed to play a sound cd in linux so i have no clue what kind 
of
support you talk about.

Last few years i was extremely happy that i could play suboptimal a few 
mp3's.

I say suboptimal as i own a professional soundcard which of course doesn't 
work
in linux, at least i didn't do much effort to get it to work there (my lemma 
nowadays is
that i don't go further than to install cdroms delivered with th eproduct, i 
no longer
am gonna do week long searches over google to figure out whether there is a 
driver
that can work) so i was forced to listen to the inferior sound quality of an 
audigy soundcard.

Of course i would have never bought that professional soundcard if it 
would've been
expensive. In this case it was 200 euro and it is a maya 1010 card.

> Still, motherboard/chipsets suffer from a lag, especially on
> motherboards that add some "differentiating" chips or features (or are
> just plain broken relative to spec).  Cameras are much better but still
> iffy, especially CHEAP cameras.  And so on -- ditto with the software
> manufacturers tend to release with their hardware.  So it has gotten
> better, but this gap is still open and very annoying indeed -- more than
> enough to keep linux out of the hands of the truly luddite or
> computer-challenged who can muddle through with Windows, mostly.
> Windows has also closed some gaps of its own in the meantime, becoming
> much more stable and much more aggressively updated, although it is
> still a virus/trojan/spyware bugfarm if installed without expensive
> add-on software watching day and night.

How about RAID10 support for my mainboard (Tyan S2881) in linux?

I didn't even dare to figure out how to setup raid10 array support in linux!

> This is probably why Microsoft is trying to attack the HPC market more
> than any other reason.  It gets them a few headlines that allow them to
> convince nervous investors that today is not yet the day that the corner
> has been turned and their market share PLUNGES.  If and when it becomes
> the perception of hardware vendors that (say) 10-20% of the desktop
> market share belongs to Linux, it will no longer be so easy not to
> provide adequate linux support BEFORE releasing new products, no longer
> be so easy not to invest the tiny bit of money needed to get YOUR
> hardware's RPMs onto e.g. Livna and tracked by kernel revision number as
> needed.  The hardware driver gap will rapidly close the rest of the way.
> Commercial software developers will at last have to take desktop linux
> seriously, and work out some way of selling their products so that
> they'll run UNDER an open source environment (where again, yum will Be
> Their Friend if they only figure this out).

[conspiracy mode on]

Microsoft is of course moving in highend because the only threat to them at 
this
moment can come from the highend efforts to support linux there.

If they obtain a big market share in highend they not only make money, but 
also
can avoid a lot of companies who serve the highend exclusively to create
linux drivers.

That means that there is less incentive to keep linux a healthy and good OS,
and therefore increases the chances for windows that any competitor of them
ever will be able to release a good distribution that can deliver everything
at 1 DVDrom that is easy to work with for the user in a user friendly way,
because drivers is the most important thing there is for an OS.

Even more than stability is one.

> Now I personally think that by the time you account correctly for all
> the linux systems in China, in India, in Brazil, when you start to count
> linux systems in University environments correctly even though they are
> installed by students directly from a mirrored repo either beside or on
> top of an older Windows installation, when you account correctly for all
> those "forced" Windows pre-installs being thrown away, that Linux might
> well have a global desktop market share of 5-10% already.  This isn't

China i doubt it. I bet windows goes for 10 cent a copy there from 
cd-recordable
to illegal reproduction.

> reflected in sales figures, of course -- most of the copies extant were
> installed for free and with no direct reference whatsoever to the
> originating company (if any).  It is all but impossible to measure just
> how many systems there are installed in this way without walking the
> entire tree -- perhaps checking the yum update repo logs all the way to
> the bottom might give you an accurate count -- but then there is debian.

> If my surmise is correct, then yeah, duh, Microsoft HAS to do ANYTHING
> IT HAS TO to keep it quiet.  If you think MS employees have underwater
> options now, imagine what they'd be a day after that news hit the
> street.  However, it is running on empty.  Software developers have
> learned the hard way that if they ever invent NEW software that is worth
> a ton of money, they'd better be prepared to give up 70% of the market
> share to Microsoft on demand, although Microsoft has backed off on that
> in recent years just a bit and let some smaller markets live unmolested,
> maybe as a result of the antitrust suits they "lost" (really, won by
> virtue of the wimpiness of the settlements -- a half-billion dollars was
> nothing compared to the years of delay and gain in market share).

Hopefully highend will be a long and hard battle for microsoft
and not one they can easily crack.

However as long as highend manufacturers keep the profit margins as high as 
they are now
on each product (usually they're forced to do so because they sell no 
quantity) then Microsoft
will have major field days.

>>
>>> From roughly 28$ to about 22$ right now.
>> So earning yourself some options to buy microsoft shares will probably 
>> not be so popular
>> among microsoft employees unlike the past.
>>
>> Some things of them are becoming real nerd products, such as their server 
>> edition 2003
>> is an impossibility to be used by normal users.
>>
>> Basically i see a few big issues that are all USB related that can 
>> crash/lock up windows
>> completely, apart from that their reliability has been greatly improved. 
>> It can run without
>> crashing for a few weeks now.
>>
>> So if they put out a 'cluster product' now, you can laugh, but basically 
>> it means money for companies
>> who produce software to run under that OS, so they WILL produce software 
>> for the cluster expert edition,
>> as microsoft WILL sell ten+  million of copies of their server editions.
>
> Ten million copies?  Vincent, if you added up all the top500 systems,
> and every system in the top 500 had an AVERAGE of 1000 nodes, it would
> still only be 500,000 systems. or a twentieth of that.  Even given the
> huge IBM clusters and Spain's cluster, I doubt there are a million nodes
> in all the top 500 put together, and downhill from that you hit
> university clusters and abject poverty.  That is, the vast bulk of nodes
> in both the top500 and everything else are in environments that would
> never, ever pay MS for software on this scale.

Majority on this planet is small companies with 1-7 persons.

I'm not sure whether you realize, but microsoft already cannot be beaten in 
that
highend area. That's obviously where the majority of their sales will go to.

They don't own 1000 node clusters.

For a part Microsoft creates their own market here.
Many of those companies will only buy a small cluster BECAUSE
there will be within a couple of years software that can work with
microsoft cluster editions.

Let's hope microsoft never takes over the 1000 node clusters,
but i fear especially there the competition will be difficult for
highend companies.

For the coming years that area will be pretty safe of course,
but for how long?

All it requires is 1 highend manufacturer of a highend network card
that wants to sell it for $100 and has a 8 node switch that
sells for <= $1000 instead of todays $400-$1000 prices a card and $3500
for a decent switch.

When a manufacturer sees he can sell it for such a price in huge
quantities thanks to having an integrated driver into windows cluster 
edition,
he'll do it.

How can the other highends compete against microsoft then if microsoft asks
in return for a faster latency+bandwidth the price of exclusivity?

Just do the math. A 8 socket machine is quite expensive as of today.

With the fastest dual core opteron that's soon nearly $30k.

Buying 16 nodes dual core single cpu with a decent network, if that has a 
hardware
price of $20k, then perhaps it will be faster than a 8 socket machine,
and sure is attractive to a small company.

Right now it isn't because linux is too complicated and the software they 
want to
run with a single mouseclick hasn't been ported to linux-mpi yet.

That entire market of 1-7 person companies, consider it already taken over 
for 99%
by microsoft.

> They aren't running GUI applications, and if their code is all MPI and
> runs just fine on Centos 4.0 or Scientific Linux or Warewulf+whatever,
> why in the world would they change?  Just e.g. getting cernlib to build
> on a system is huge piece of work that has already foiled porting many
> of these applications to anything but SPECIFIC versions of SPECIFIC
> linux distributions -- porting costs are a major obstacle now as ever.
> These folks KNOW that it isn't cheaper to run a Windows network than a
> linux network, ever.  They KNOW that you cannot possibly install an

It is cheaper to run a windows network.

Hiring a linux sysadmin fulltime is a huge salary cost a year for 1-7 man
companies.

A windows cluster is way easier to maintain...

If we can agree upon that the entire small companies market already goes to
microsoft entirely, with just Sun as a competitor of microsoft, because Sun
offers higher RELIABILITY than microsoft, which to many companies for
now is an important criteria to pay loads of money to Sun, then you might
perhaps finally start to realize the problem for highend companies in
the future.

If everyone grows up with windows clusters, after a while starts to trust 
them,
and knows how to WORK with them, i'll have to see how long
some commissions can block out windows and pay more for a linux cluster.

Perhaps it happens... ...if IBM keeps delivering good supers... who knows?

> operating system in bulk quantities more easily and cheaply than is
> possible with Linux right now, which is free (per seat/node) and at the
> direct expense of a minute or so of labor per system, if that. They KNOW

Yes let's talk about the labor per system.

How many sysadmins get paid to maintain a linux supercomputer and what do
they earn a person?

> that their MCSEs are overpaid and clueless about clustering (unless they
> are bright and have been re-educating themselves on the Linux clusters).

Aha the first dark clouds gather above your university.
You need to be BRIGHT and INTELLIGENT and re-educated
to know how to work with linux clusters.

So let's say 2 times the salary of a windows cluster admin?

So you ship a bus to Mexico, let it come back with 40 persons.
20 get stopped at the border.
The other manage 20 come to get through somehow.
You give them a windows course.
19 of them succeed for the course. 1 of them left prematurely as he was
intelligent and goes work at the competitor, namely some other university
with $100k sysadmins and IBM linux supers.

The 19 lucky bastards you hire to become a $10 an hour sysadmin of your
uni, earning a person $30k a year.

Your savings a year: $70k * 19 = $ 1.33 million

Hmmm, makes windows quite attractive for big supers!

> There might or might not be ten million cluster nodes in the world, but
> MS would be very, very lucky to one day, if they work very very hard and
> discount their HPC product to basically "nothing" and/or focus on
> commercial markets as previously noted, end up on 10% of them.  Nothing
> being precisely the marginal cost of most of the products they are
> competing with...

You forgot that 95% of 1-7 person companies?

> Price DOES matter, in the long run.  In the cluster world, Solaris has
> pretty much precisely the same advantages and disadvantages as Linux
> does -- except for cost.  Indeed, arguably most of the original

Right, i see a $ 1.33 million reason a year to move to windows.

> development of open source PVM and MPIs occurred under SunOS, as it
> dominated the workstation market at that time.  Great support from true
> experts, great market reputation, decent hardware performance wise, and
> a huge head start.  Now count the number of Solaris/Sparc clusters in
> the world compared to Linux/Intel/AMD.  Oooo, guess people DO care about
> that factor of two or more in price/performance... really really care.

> Ultimately, I agree with the earlier assessment (from Jim?), that the
> entire HPC market is no more than a pimple on MS's behind as far as
> likely contribution to MS's bottom line is concerned, and suspect an
> ulterior motive in their entering the market -- like trying to at long
> last put a finger in a gradually widening crack in the leaky dike that
> supports their product all nice and dry and monopolistic and dominant
> inside its protected ring of FUD.
> But it's an ocean out there, exerting a quiet and inexorable pressure on
> that wall.  The wall is crumbling, bit by bit, and every bit that washes
> away is very, very unlikely to ever come back.  One day, possibly soon,
> there will come a storm....

When lying at the beach at the ocean perhaps Jim has a bit more time to
reconsider what the difference will be when the sticker on his machines
reports 'wintel inside'.

It's just 1 letter difference with the current situation there...

Vincent

> Regards,
>
>     rgb
>
>>
>> Vincent
>>
>
> -- 
> Robert G. Brown                        http://www.phy.duke.edu/~rgb/
> Duke University Dept. of Physics, Box 90305
> Durham, N.C. 27708-0305
> Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu
>
>
>