[Beowulf] MS HPC... Oh dear...

[Robert G. Brown]

On Tue, 13 Jun 2006, Vincent Diepeveen wrote:

> You and i are like junkies busy downloading all kind of linux distributions 
> 'for free',
> sitting and waiting a LONG time (in your case expensive time paid by the 
> government,
> in my case i just CLAIM my time is expensive as i'm running my own company).

Oh, I do companies from time to time as well, don't worry.

> Then we burn some DVD's and CD's in order to find out after a few weeks of
> trying that Ubuntu nowadays is the best distribution. The rest is utter crap 
> (from the
> FREE distributions) now that we have 64 bits dual core monster chips.

I wouldn't exactly call FC5 utter crap -- he says, typing away on an FC5
box.  FC4 wasn't crap either.  Ubuntu is arguably a hair more bleeding
edge current these days.

However, I personally start a mirror rsync on a new FC distro one
evening, come back the next day sometime (I have DSL so it does take 24
hours or thereabouts to get multiple GB at 2 MB/min or so) and then can
(re)install all the systems in my house in a matter of twenty minutes or
so each, rate limited mostly by bandwidth and the CPU required to unpack
all those RPMs and write them to disk.  Since most of those systems are
kickstart installed, getting them ready to go is a matter of fixing
their DHCP reference, tweaking tftpboot to add a path to the right
kickstart, and picking a target of the boot list.

To get windows (not being a thief or a pirate:-) I have to get in my
car, drive for twenty minutes, spend $200 US for a box set of XP Home
edition, drive twenty minutes home, then spend the next DAY trying to
get the damn thing onto a system, configure the hardware, remove all the
crap advertisements, "free" AOL trial memberships from hell,
complimentary McAfee or Norton subscriptions.  I have to visit Duke and
put (fortunately) site licensed (and legal!) copies of McAfee, ssh, and
a few other tools on the system so that I can actually remote access
systems I work on.  If I fail to do antivirus, of course, it is a matter
of weeks before I have to either do a clean reinstall or hope that I can
clean the system ex rape facto, and naturally it is a race to change all
my passwords, including ones used at e.g. my bank or paypal, before the
id-theiving trojans communicate them back to the rapeware on a russian
server somewhere in the middle of nowhere the law can reach.  If I
>>do<< the antivirus, all I really get is an extension of the time until
this is necessary, as Windows AV is always behind the curve by weeks and
weeks, not hours and hours.  I have to register my copy of Windows, turn
on automated updates, monitor those updates while they install a few
service packs (or else not even AV can save me).

If I'm lucky and all the stuff I de-install doesn't actually break
anything in the process, I can then use my brand-spanking new windows
system -- for pretty much nothing.  But browsing the web and perhaps
reading email, if I don't mind using a tool that has been a bleeding
security wound for many years now, of course.  To do ANYTHING but web
browsing, mail, and playing minesweep I have to go buy software, and the
first piece I must buy is Office.

Oh dear.  Another trip back to the store, this time to drop even MORE
than $200 US -- as much as $400-500 US if I want Office Professional.
Home again, jiggety-jig, and I install the fat pig on my Windows XP Home
edition system.  Finally, I can actually word process, login to remote
places with ssh (or, god help me, use Windows Remote Desktop to directly
access a desktop on a friends remote box if I don't mind sucking down my
entire DSL line's bandwidth crying for more and my friend doesn't mind
bending over and spreading his system's metaphorical cheeks for my
intrusion).  I can read my main with Outlook (whoopee!).  I can browse
the web with Explorer, or even download and use Netscape instead if I
don't mind breaking a few things.  Of course I can't REMOVE Explorer or
the system will explode and I'll have to reinstall...

This isn't all imaginary, of course.  I do some work with Windows as
well as Linux (regretfully and resentfully).  I will cheerfully wait 24
hours to download a COMPLETE IMAGE of an installation repo and then
install a cron job to keep it up to date and then install linux boxes
from PXE flawlessly and unattended EVERY TIME compared to that.  And
yes, I'm aware that Windows can be installed over PXE as well.  I'm also
aware of how much work it is, per system, and the number of places a
human has to make decisions in the process.  It isn't exactly an
unattended, fully automatic (or automatable) process, unless they've
improved it from when I last looked, which is certainly possible.

When the linux system comes up, it is secure.  Period.  Perhaps it is in
principle crackable, but only by der Ubercracker, not by any of the
eighty zillion Fully Automated Viral Spambots crackling across the net
at any given instant, not by any of the umpty umpty booby trapped
websites that install spyware that opens trojanware that permits
virusware onto your box, and then proceeds to eat your lunch and piss on
your shoes.  I actually cannot remember the last time Linux was
massively exploited -- some sort of exploit that hit only sites running
an non-updated Apache, IIRC, maybe four or five years ago?  Either
pre-yum or not long after -- too many unprotected sites without updates
permitted the exploit to get a bit of traction before being yummed out
of existence.  Nowadays a cracker has at most HOURS -- perhaps one whole
day -- to use an exploit before the hole is closed nearly everywhere.
No room (epidemiologically speaking) for growth in that.

> OTOH sure microsoft undersells in practice. The average Joe walks to his or 
> her neighbour
> and asks for a free cd recordable from his neighbour with 'windoze' at it.
>
> Directly without lifting a single eye brow, the neighbour hands through the 
> door a cdrom.
>
> Dang we've been beaten in price BIGTIME then already, not to mention TIME.

Sure.  Can't argue with theft and piracy.  However, you can do exactly
the same thing with linux CDs, only nobody bothers.  A CD is a static
image -- a repo has both the static CD (basic install) image AND the
updates AND extras AND livna AND etc...  The only advantage to having a
CD is the bandwidth -- it is faster to move a few GB around that way
than on a wire.

Of course with Linux reproducing, even MASS producing those CDs is
legal.  Sell them, give them away -- legal.  With Windows you are BOTH
violating the law AND behaving unethically.

>> You also missed another of my points.  It has been possible to write
>> parallel software that runs on Windows boxes since maybe 1993 (can't
>> recall the exact date that somebody did the Windows port of PVM, but I
>> vaguely recall seeing Windows ifdefs in the source about then).  There
>> have been plenty of groups with many Windows-based desktops available,
>> sitting nearly idle 90% of the day, pretty much forever.  These systems
>> have always been "free" in the sense that they are already there and
>> paid for and are sitting idle.
>
> I remember having a discussion a year or 13 ago with a professor who said 
> that his university
> always would have sun or hp boxes. His systemadmin wasn't happy with that 
> statement and
> semi publicly complained about it. He just saw that for 130 students there 
> was just 20 sun boxes.
>
> With PC's it would've been possible to put 60+ boxes there at the time, 
> besides that they were
> faster also, about 3 times faster objectively. On paper of course Sun and HP 
> had great whitepapers
> supporting a good speed and a low price for those boxes for each thing 
> achieved.
>
> Standardizing for the masses *always* wins in price from all those small 
> dedicated solutions
> that eat money and extra service personnel as the hardware is too complex to 
> understand
> for the average joe.

Agreed.  Which is precisely why those sun boxes AND windows boxes are
largely replaced at Duke with Linux boxes.  You absolutely cannot beat
PXE+Kickstart on a high-quality commodity box with onsite service (and a
low probability of failure in the first place) for running student
clusters.  Installs in a few minutes each, unatttended, reinstalls
(remotely initiated or not) equally unattended, comes up ready to run
without human intervention, contains a completely standardized set of
packages both free and commercial site licensed, supports a strong
authentication/security model, supports fully encrypted network access
to most major services, supports industrial strength remote file
services with decent access controls (at a ratio of HUNDREDS OF SYSTEMS
per server, not FIVE SYSTEMS per server).

>> the cluster market has not been terribly tolerant of cost-inefficiency
>
> Yes the supercomputer market is pretty cost inefficient.
>
> If it stays like that, then you can vote in that commission for an expensive
> linux supercomputer, if it doesn't then obviously the doom scenario i
> wrote on the upside of the beer mat is gonna happen.

Um, this is BACKWARDS.  The cluster market is INTOLERANT of
cost-inefficiency.  Right now it is so efficient that it screams -- you
typically pay for the hardware (unavoidable and the same for any
possible OS), the systems management (where linux outscales MS-anything
by a factor of at least 3-4 in terms of FTEs), server resources (where
linux outscales MS-anything by a factor too large to even mention, given
that "Windows Server" is really a complicated joke that will one day be
exposed on Comedy Central as having originated with the cast of Mad TV,
programming support, where I'm sorry Jim, but coders who use gcc (for
example) have a huge advantage mentally in constructing parallel codes
on top of ANY library/coding environment that I know of -- I disagree
that event-loop coding in any way prepares one for true message passing
parallelism and the contemplation of scaling issues, and is barely
adequate for distributing effectively EP blocks of work IF one has
figured out how to do threads that execute concurrently with a GUI event
loop and are sufficiently state aware that they can run and communicate
safely with the GUI. Most programmers never really figure that out, as
it just isn't what GUI apps are all about.  And it isn't, actually,
terribly easy (he says as he works on a GUI app with a concurrent
execution thread, requiring the usual state-awareness, variable locks,
and all the rest).  Finally, there is the cost of the OS, libraries, and
"other" software, which is free in the case of linux and very expensive
for Windows.

I mean, folks keep talking about that fully burdened engineer who wants
to access the 1024 node cluster, but what about the fully burdened team
of eight full time Windows admins needed to ADMINSTER the cluster (and
that's 128 nodes each, which is pretty damn many for typical Windows
shops!).  What about the god-knows how many servers required to make all
those nodes happy?  What about the fact that several hundred dollars a
node in software adds up to several hundred thousand dollars, which is
enough to appear on many a radar screen even in spend-happy
environments?

The linux based compute cluster was invented to BE the most cost
effective supercomputer in the world.  That's what the Bell prize is all
about.  Is there a single human on the planet who thinks that a
Microsoft cluster designer is going to be a candidate for the Bell
prize (if real costs are reflected in the design)?

> There isn't a single serious compiler for windows left other than
> visual c++.

Odd.  The only tool I ever hear of being used to design MS software
in-house is VB.  VC++ is used primarily by software developers that make
commercial stuff, and I'm not sure that it is the PRIMARY tool of choice
even there.

No accounting for taste, right?

> Why?

Got me.  I don't particularly like C++.  If I had to guess it would be
because MS screwed Borland to the wall shortly after they invented Turbo
Pascal by yanking the OS around in such a way that several generations
of Turbo C++ broke but Microsoft's C++ did not, so that developers were
forced to abandon TC++ or lose market share.  But that's only a guess,
not an actual assertion.  Maybe it was accidental.  "Hey, we were just
lucky I guess".

>> So, perhaps they've finally identified that ideal rich-but-stupid
>> segment of potential cluster customers that can make them high-margin
>> money; perhaps they've decided that they have to get into the market
>> even if it is a dead loss forever or lose market share elsewhere, who
>> knows?  We'll see how long it takes for them to buy themselves a top 10
>> cluster somewhere, like Apple did a few years back.  Did the apple
>> cluster materially affect the dominance of linux/x86?  It did not.  Wil
>
> You should really start worrying here.
>
> Microsoft in past also copied Apple and really grew big with those
> stolen ideas.

Why should I worry?  Apple made a huge number of very basic mistakes.
They are only this year, slowly, painfully, ALMOST acknowledging that it
is dumb for a software company to pretend that it is a hardware company.
Sun microsystems has been spanked for the same reason, almost out of
existence (any decade now:-).

I'm fully aware that Microsoft is a viscious and unprincipled corporate
entity that will do whatever it thinks it must to maintain its market
dominance.

However, the linux model is one where their usual tactics Will Not Work.
I promise.  They absolutely rely on high margin sales, big profits for
relatively little added value.  Linux intrinsically relies on a service
model where the added value is all you pay for (on the commercial side).
Linux inexorably advances, and advances via a programming model that is
intrinsically more innovative than Microsoft's.  Linux is (IMO) drawing
near to Microsoft, catching up to them in the key dimensions, and once
they do Microsoft now and forever will be eating Linux's dust as
Microsoft CANNOT do the same things to Linux that it did to IBM, to
Borland, to Lotus, to Netscape.  There's nobody to do them "to".  They
cannot undercut Linux's prices.  They cannot develop new products faster
than they are developed on Linux.

They have to be very careful now about antitrust suits -- they've
narrowly escaped (lost the suit but barely maintained the status quo
stalemate) several times now.  They are fighting hard to create some
sort of controllable IP in webspace but there they are truly handicapped
by the incredibly poor cost scaling of their server products and the
fact that nearly all developers understand the need for open standards.
I'm not worried about the viability of Linux.  I'm not worried about the
viabililty of Linux-based compute clusters as a dominant model for
cluster computing.  Microsoft will do its best to play unfair and dirty,
and it won't matter, because unfair and dirty will still leave it
EXPENSIVE.  Ask Sun Microsystems about that -- does expensive work, in
the long run, compared to not expensive?  It does not.

I personally think that the real war will be fought, is being fought, in
a few very limited regimes of functionality.  Ease of installation and
power of the result for novice users, where Linux is pretty much tied
with Windows (unfortunately, tied at a pretty horrific level of poor).
Windows has the advantage of pre-installation on most systems, that's
about it, but Linux is the one that is trying to make installation
EASIER.  Eventually they will succeed -- the tools to do so are already
there, really, and being serious developed to perfection.  Microsoft
will continue to rely, I'm sure, on controlling distribution channels,
but that only works for consumers and works less efficiently as time
goes by and people get sick of paying MS $100's of REAL dollars for
"upgrades" or are informed that $100+ of their original system price
was, in fact, Windows.  And then there is Office -- HUNDREDS of dollars
more and NECESSARY even at home.  Desktop functionality for home users
and for well-managed LAN users is equal to or greater than what MS
provides without exception now, and is as easy to install EXCEPT that
this still isn't easy enough.  Device drivers continue to be a problem,
but there are signs that even this problem is drawing to an end.  MORE
vendors are support linux, not LESS.  Hardware vendors are continuing to
figure out that Linux really IS a viable market that contributes
nontrivially to their bottom line.

Business middleware.  Here I agree that linux is behind the curve.
Don't know what to say.  If my current business enterprise makes me rich
enough, maybe I'll fund a company to fill the gap.  This really
surprises me -- if Red Hat had a lick of sense, they'd have a whole
division of full time programmers doing NOTHING but writing a whole
perfectly integrated suite of business middleware, all fully GPL, all
moderately customizable on the front end, all hooked into high quality
scalable DBs on the back end.  The whole nine yards -- POS, inventory,
accounting, HR, time cards -- most of the UIs web-based and
cross-platform compatible so it would support WinXX clients as well as
LinXX clients, and secure.  Call it "Red Hat Business Suite", GIVE away
the software and sell service contracts on it according to the scale of
the business using it, discounted for businesses that run it on RHEL.
Price it so that it is cheap, support it so that it is reliable, and it
would kill off WHOLE INDUSTRIES on the Microsoft side -- worth it as a
loss-leader MICROSOFT killer.

Believe me, I sit on the board of a small business, and this would sell,
sell, sell.  Small businesses HATE buying Microsoft server products with
their huge human admin and maintenance costs, their extraordinary
software costs, the high cost per seat.  These are a myriad of
businesses with small profit margins and spending a few tens of
thousands on IT is a lot for them.  You can spend that for an accounting
package alone as it is.

>> Microsoft's playing exactly the same game make any significant
>> difference?  I honestly doubt it.  And who is going to help them?  IBM
>
> Wintel?
>
> For some years i've had this nightmare already of a hardware chip
> that just works for windows.
>
> Just because intel is making quite some billions a year,
> m$ can't really make a clear appointment there i bet.
>
> How many companies in highend make several billions a year?
> Sun, IBM...
>
> At least one of all those companies in highend will grow real big by
> making an exclusive deal with microsoft.
>
> So as long as you are willing to buy an IBM or Sun platform,
> you might be able to keep windows outdoors.
>
> I do believe however in a wave effect. Sometimes a company makes
> a good product, then next year some other manufacturer has a better one.
> And a few years later your wave peeks again and you have the best
> product again.
>
> If that's the case then you'll have to buy sometimes really shitty hardware
> to keep linux.

Intel isn't that thrilled with Microsoft, and the antitrust suit that
this would bring about would NOT be survivable by MS.  They'd get broken
apart.  Intel wouldn't be too happy with it either -- they JUST managed
to sell Apple on using Intel chips.  Intel is all too aware of what
happened to IBM, as well.  If they help MS become TRULY monolithic, they
will become a de facto MS subsidiary.  They aren't eager for that to
happen.

> It's not a matter of you not being prepared to pay only.
> There is no 'linux only supermarkets'. (where supermarket = big distributors)
> Supermarkets are willing to put certain windows products there.
> *that's* why windows software sells.
> BECAUSE it is in the supermarket.
>
> If supermarkets would accept linux products, i would of course
> hand them over a linux product.
>
> GCC is for free, and open-gl also works fine at linux.
>
> So i can cut costs to produce a linux version.
>
> But who is gonna sell it for me?
>
> Products sell because users see the product.
> If microsoft is going to push a cluster windows version,
> then i'm basically busy with: "how can i produce a product
> that works great for cluster windows, because tens of millions
> will want to get that cluster windows, all i need to do is sell
> them an app they like to toy with".

Software shops here will certainly accept Linux products to sell if the
margins are right.  Bookstores are also a reasonable venue -- include
the software as part of the manual, sell the manual as a "book" at
Barnes and Nobel.  HOWEVER, I personally would advocate selling
linuxware on the web, using paypal to manage the transaction per se, and
yum+authentication to permit access to a repo.  Customer pays you, drops
an rpm onto their system from your website that contains a private
authentication key that matches some system-specific token, and
thereafter can do "yum install superchess" and have the program
auto-update nightly in cron from your site, facilitating upgrade/update
and maintenance.  Every year, every six months, they pay you again
(subscription service) or not if they're happy with the version they
have running.

Sigh, look, I just invented ANOTHER new product for linux that could
completely revolutionize software distribution and incidentally wound
Microsoft beyond compare.  Linux is at this point fully capable of
supporting a completely automated online software shop, where you don't
have to move from your seat to shop for and purchase and install and
automatically update and/or get service/help for rpm packaged software,
per distribution, for the major distributions.  Developers haven't quite
twigged to it yet, but yum can potentially revolutionize COMMERCIAL
software distribution.  Don't just buy the software, buy precisely the
right version of the software to match your system and get instant,
free, updates and bugfixes.

The nice thing about this distribution channel is that it doesn't cost
much.  Paypal is cheap (way cheaper than the margin any retailer will
charge).  A website is something you're likely to need anyway, and any
website is a potential repo.  If it works at all, it pays for itself.
Google becomes your friend, referencing potential clients to you if you
craft your site metadata well.  An associated mailman list gives you
direct access to your clients and a service channel.  If it is an open
source product, you can run a CVS tree.

> To mention one simple problem under linux; fonts picked by designers
> that were expensive paid and look great in windows, if you display that
> same thing in linux, somehow linux messes it up there. The font is of
> other size and looks different.

You've installed the Microsoft font package?  For Linux?

>> likely -- or build it as a PHP or java or Gtk app on top of your API.
>> This really divorces the choice of end-user platform from the actual
>> compute engine/cluster, and lets you focus energy where it makes sense,
>> probably in the latter once you have ANY sort of simple GUI running that
>> can talk to the engine.
>
> JAVA was a nice try from Sun to accomplish exactly that.
>
> JAVA is too slow for games however.
>
> We are busy with realtime software.
> Speed matters.

Speed matters on the BACK end, not the FRONT end.  GUI speed is fast
enough ("instant") if it is <0.1 seconds, usually.  I thought there
already were chess engines that have a well defined network API, such as
xboard vs internet chess servers, crafty, gnuchess, etc.  Why not just
make your engine use xboard, an already extant perfectly functional GUI
for the gaming side?  The most you'd need then is an engine
configuration GUI, which could be web based.

> The videocards are so slow anyway of the average pc, that we'll need quite a 
> bit of
> system time to render 3d scenes.

Yeah, xboard isn't 3d.  However, it would be easy enough to add 3d to
the engine I'd guess.  Certainly wouldn't hurt your program sales per
se, since the front end is in a sense irrelevant to the real value in
the back end.  But this is getting OT.

> Let's do you a public offer.
>
> If you donate me 10000 dollar i'll port our 3d GUI also to linux
> and start selling it for the same price like the windows version.
>
> The port will cost me more in fact as support for each version will
> eat way more than $10k in human resources, not to mention the
> designers and programming time it takes us (we are with 2 programmers
> fulltime and 1 parttime). I'm willing to take *that* risk.
>
> The first supermarket boardplaying 3d product for linux.
>
> Think about it. You could make history here.

I don't play chess, or care much about playing chess, sorry.  However,
if I did and 3d boards really mattered to me, I'd render some pieces and
hack away on xboard.  It already has the basic interface, the ability to
choose the actual move engine, and so on.  It does look like the widget
set could be updated, so I'd be inclined to replace the GUI per se with
a Gtk one drawn with glade.  Oh, wait.  That would be glchess, wouldn't
it?  Of course then I'd have to learn python to hack, but hey,
eventually I'll probably have to anyway.

As far as MASTER level chess is concerned, I'm not sure how big that
"market" is windows or not.  There are an abundance of chess programs
that will beat the average casual chess player out there already, some
of them all lovely and rendered and everything.  The player GUI seems to
me to be a lot less important -- and valuable -- than the engine, and
people willing to pay for an engine better than one that can probably
beat them anyway?  Dunno.

>
> Feel free to do some postings online. I'm sure you'll find a 100 users
> willing to donate you $100 money to have that port done.
>
> I bet 90% of those users doesn't even know how to manage
> to put the shared memory bigger in linux (you need root access
> to set it from 32MB to something larger and then you have to
> explain to them what root access is). Just that thing already
> will cost me $10k to support i bet, emailing the faq having
> that info...

???

As happens all too often, I don't quite follow you here.  There is the
play engine, which doubtless needs lots of memory, all accessible from
the kernel via malloc.  Then there is the GUI, which could be, indeed
should be, a completely different program and where you don't even have
to write one -- in principle you can write to have a choice of existing
ones.

Am I missing something here?  Who cares how much shared memory you can
allocate?

> With linux you have the GARANTUEE that if you install something new
> that the *.so system will mess up. In windows at least you've got a fair
> chance that it doesn't break down as they tested it real well.
>
> If linux would have as many big apps with their own *.so libraries like
> windows has, then in general people would not be so happy about
> linux and drivers/library support.

This is not true if you work within distributions properly.  It is only
true if you try to move shared libraries between major revisions, and
then only sometimes.  Yes you can live in DLL hell, but only if you
choose to.

> Both windows and linux have done this in a crappy way.

No, Linux does it well, if you do linux right.  Windows too, probably.
The problem is that software vendors don't follow the rules.  Too much
trouble.

> A huge advantage windows has in general over everything else
> is that they really test things very well at windows headquarters.

ROTFL.

> Now that my political skills seem to improve with linux folks,
> the second version of the third email tends to get more seldom.

Hmmm, a message there I'm sure;-)

>
> Well i have some important statistics for you.
>
> The vaste majority of companies in EU is 1-7 persons.
>
> In fact roughly 95% looks like that.

That is plenty.  Too many, even.  I could do it alone in a year.  With
that much help maybe two years.  With one or two really good partners,
maybe six months, probably nine, as a corporate effort.

However, GPL projects work differently.

> Well i never managed to play a sound cd in linux so i have no clue what kind 
> of
> support you talk about.

You have my deepest sympathies.  There are no more than a half dozen
tools that will do this, all of them functional.  Then there are several
tools to master CDs, all of them functional.  Then there are the file
rippers that transform my entire CD collection in ogg format so that
they all fit (unencumbered) onto my computer to be played instantly, and
the add-ons that let me play other formats like mp3 (encumbered) and
shn, flac etc.

> Last few years i was extremely happy that i could play suboptimal a few 
> mp3's.
>
> I say suboptimal as i own a professional soundcard which of course doesn't 
> work
> in linux, at least i didn't do much effort to get it to work there (my lemma 
> nowadays is
> that i don't go further than to install cdroms delivered with th eproduct, i 
> no longer
> am gonna do week long searches over google to figure out whether there is a 
> driver
> that can work) so i was forced to listen to the inferior sound quality of an 
> audigy soundcard.

Don't know about that.  Soundcards can be a problem.  However, there is
a rather large list of supported ones.  Mine just work, pretty much,
with FC5.  FC4 won't manage the newer ones (released since FC4 was) and
so on.  Don't know how to do much about that.

> How about RAID10 support for my mainboard (Tyan S2881) in linux?
>
> I didn't even dare to figure out how to setup raid10 array support in linux!

Don't know.  I use md raid.  Onboard raid tends to make me nervous
anyway.

> China i doubt it. I bet windows goes for 10 cent a copy there from 
> cd-recordable
> to illegal reproduction.

China actually pushes linux as a high level decision.  Microsoft has to
formally complain about piracy, and this embarrasses them.  They are
also not fools -- they understand what the word "monopoly" is all about.
Only the existence of alternatives keeps MS from instituting a crypt
handshake that makes piracy impossible.  That is to say, it is "public
relations".  They need good public relations only as long as there is
something for the "public" to choose.

> It is cheaper to run a windows network.
>
> Hiring a linux sysadmin fulltime is a huge salary cost a year for 1-7 man
> companies.
>
> A windows cluster is way easier to maintain...

Glad to know it, given that you haven't seen or touched the product.
Windows isn't easy to maintain period.  Windows networks certainly not.
Linux isn't terribly easy to set up or run if you aren't semitechnically
inclined and mentally competent, but SOFTWARE companies tend not to be
founded by people who aren't.

> If we can agree upon that the entire small companies market already goes to
> microsoft entirely, with just Sun as a competitor of microsoft, because Sun
> offers higher RELIABILITY than microsoft, which to many companies for
> now is an important criteria to pay loads of money to Sun, then you might
> perhaps finally start to realize the problem for highend companies in
> the future.

Don't be silly.  Right now a small company can run linux for the
investment of human time and the inevitable cost of machines.  With
Windows to set up anything like a professional environment costs
thousands of EXTRA dollars that come out of pocket.

People do Windows development only because they want to sell software to
run on Windows machines.  They dream of making it "big" selling that
software.  However, more and more developers are familiar with the
oft-demonstrated life cycle for Windows software.  Invest money and
sweat and brains, invent something brilliant, develop the market for it,
and then prepare to either be bought out by Microsoft for roughly what
your next year's worth of profits would have been, with the alternative
being that Microsoft clones your product in three months of intensive
effort and systematically co-opts 80% or more of the market YOU
developed.

Believe me -- if computer chess were really mass market valuable and you
invented the universe's best mass market chess program, the ONLY mass
market chess program, you would hold the market you developed for six
months after Microsoft decided they want it.

I mean it is really very simple.  Microsoft sells the operating system,
the software development tools, consumer client software, web client
software and server software.  They have a huge pool of world-class
programmers, many of whom are semi-idle or redirectable at any given
time.  They control the strongest marketing channels in the known
universe.  They have immense cash resources.  They can in the future as
they have in the past take over any product that exists with an
advantage so huge that calling it "unfair" is a world-class
understatement and either put the inventor of that product out of
business or reduce them to an afterthought in the market.

The truly amazing thing is that Microsoft has managed to create a
universe where the prospect of creating a new product so you can be
bought out by them is considered a DESIREABLE end for the developers.
At least that way the developers get rich, kind of, once they've taken
all the risks and so on.  They don't make anything like the money they
deserve, but hey, who needs to be REALLY rich?  Bill Gates, of course.
Not you.

If you develop using Microsoft as a platform, you are either:

   a) Developing for a one-off application, as Jim was describing for
inside a company, no commercial potential;

   b) Developing for a small niche market, one that will never pay
Microsoft for the cost of cloning your product, or writing games, which
aren't clonable (Microsoft writes its own in competition, of course, but
they can't copy the game per se and branding matters in the game market
as do reviews).  So games are pretty safe and are even an architype for
the kind of thing that is safe.

   c) Developing for Microsoft.  You may THINK you're working for
yourself, but you're really just part of their vast unpaid staff, hoping
to succeed and be rewarded as any good employee is, with enough to
retire comfortably and a gold watch.

No wonder that a lot of the really good, hotshot developers think
seriously about developing REALLY innovative stuff either on the web (a
neutral platform difficult for MS to co-opt however much they've tried
and continue to try) or yes, for linux.  The open source paradigm is a
tough business model, but an ASP >>service<< business model works for
many products, within the GPL.  The GPL only matters if you sell the
software, not its services, at least unless I've read it incorrectly.

> Yes let's talk about the labor per system.
>
> How many sysadmins get paid to maintain a linux supercomputer and what do
> they earn a person?

As I said, it would help if you actually paid attention on this list.
There is no organization on the planet where maintaining linux on a LAN
basis or server basis costs more than it does for Windows.  For clusters
that is true in spades.  With a Linux cluster or LAN the number of
systems one admin can care for is limited by the requirements of
providing user support (help desk) and dealing with hardware problems.
In a desktop LAN, well over 100 systems and their users per admin, often
over 200.  For clusters, well, Mark is taking care of what, 500+?  Lots
of people I know are caring for hundreds of nodes with one admin.
Surely you aren't asserting that this will work for Windows HPC, with a
ratio of 10:1 nodes:server or thereabouts.

> Aha the first dark clouds gather above your university.
> You need to be BRIGHT and INTELLIGENT and re-educated
> to know how to work with linux clusters.

C'mon, you're really not from Earth, are you?  Maybe you haven't noticed
the high school students who've made their own clusters with a tiny bit
of help from the list.  Wake Tech over in Raleigh has a "cluster tech"
training program in place -- I'm sure its students are bright enough,
but we're not talking rocket scientists.  I've advised undergrads in
India, Brazil, Spain, and several other countries through building their
initial linux clusters.  I mean for gosh sake, Vincent, to build a
cluster:

   a) Figure out how to install your favorite flavor of linux on a
desktop system with maybe 10-20 GB of free disk (an absolutely trivial
amount these days) hooked into a switched ethernet.

   b) Install warewulf, following its online instructions.  In many cases
straight from pre-configured packages.  Install whatever you like --
pvm, mpi etc -- in your exported node image.

   c) Configure warewulf to support your nodes.  This is pretty much
putting their MAC addresses in the right place so that they will
dhcp/pxe boot.

   d) Boot the nodes.  Diskfull, diskless, who cares.

Poof, it's a cluster.  Have fun.

MOST clusters, especially development clusters, run on plain old
ethernet.  Can do a lot of work on ethernet.  If you want or need a
faster interconnect, you have to pay for it, sure.  You may well need a
higher class of admin (or not -- it isn't THAT hard to figure out and
this list will help ANYONE).  But that's the way it always is -- the
more you need the more you have to pay.  Don't think for a minute that
it will suddenly be easier for Windows so that any high school kid can
do it, and if it is, heck, any high school kid can do it NOW.

> So let's say 2 times the salary of a windows cluster admin?

Do you have any data to support this?  Why do you think this will be the
case?

> So you ship a bus to Mexico, let it come back with 40 persons.
> 20 get stopped at the border.
> The other manage 20 come to get through somehow.
> You give them a windows course.
> 19 of them succeed for the course. 1 of them left prematurely as he was
> intelligent and goes work at the competitor, namely some other university
> with $100k sysadmins and IBM linux supers.
>
> The 19 lucky bastards you hire to become a $10 an hour sysadmin of your
> uni, earning a person $30k a year.
>
> Your savings a year: $70k * 19 = $ 1.33 million
>
> Hmmm, makes windows quite attractive for big supers!

You leave me speechless.  Literally.  I mean, I cannot even.... no, I
just can't.

Vincent, you've finally convinced me.  I give up.  You're right.
Windows is irresistable.  Bill DESERVES my money -- after all, he gives
to charity all the time, right?  I'm going right out with my Visa card
to spend the $5000 or so it will cost to convert my house over to
Windows-only and only give up a bit in functionality, and get a charter
copy of Windows HPC to run along with the two copies of Windows Server
I'll need.  Then I'll go hire a Hispanic person without a green card to
manage my household network/cluster.

Damn.  Why didn't I think of that before?  I might as well have them
write software for me while I'm at it, and maybe clean my house.  I
mean, $10 an hour is really cheap, right?  And I probably won't have to
pay social security either.  And hey, I even speak Spanish!  This is
just ideal!  Thank you thank you thank you.

I'll have to pass this idea on to Duke, it's just BRILLIANT.  Right now
they are foolishly paying a small fortune for Windows adminstration of
various clusters, and the campus security czar sweats blood waiting for
the next major exploit.  And heck, they use Linux all over the place
and that is SO obviously just WRONG.

I'll bet that if we put Windows EVERYWHERE we'll save LOTS of money
especially if we use illegal copies and illegal labor as per your
suggestions!  I'm sure the US Immigration service won't mind, and the
IRS -- well, heck, they ignore all the migrant farmers and construction
workers, don't they?  It's only fair.  And hey, Microsoft is bound to
let us steal their software with a wink and a grin -- they KNOW that we
train the next generation of Windows users and coders, although with the
new Mexican resource that may no longer be true.

Vincent, I hope that you are using this method for the software you're
developing -- but I'm sure that you are.  In Europe maybe it isn't
Mexicans, but surely you can outsource to India -- everybody else does
(even though this of course is legal).  You can just give them a
specification for a really fast chess engine and tell them to make it
work with Windows HPC and voila!  It will be so!  At a few rupees and
hour -- a pittance.

Listen everybody, Vincent has finally opened my eyes.  This list is
hereby obsolete.  Linux is too -- it always has been, relative to
Windows.  I was a fool to leave DOS and Windows 3.2 for linux those many
years ago.  So this will by my last posting.  Maybe Windows will sponsor
a list for Windows HPC.  I hope so -- although with all those Mexican
Windows sysadmins out there doing clustering for cheap without anything
like a technical education -- well, what would anyone ever need ME for.
Maybe I'll take up writing bad science fiction instead.

    rgb

(Desculpame, por favor, todos los gentes hispanicas en esto listo.  Pero
el subio no es serioso, es loco.  Yo creo que si.)

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu