[Beowulf] Why I want a microsoft cluster...

[Joe Landman]

Jim Lux wrote:
> So here we go with some devil's advocacy...

:)

> 
>  From the user viewpoint, in a largish shop, but with a single user in mind
> 
> The scenario is that I want to run some sort of analysis tool that is 
> computationally intensive enough to require more crunch than I can get 
> with a single desktop.  Applications that spring to mind are various 

[...]

fine so far

> So, whatever I do, my output is eventually going to wind up pasted or 
> copied into some MS product, AND, it has to be "clean" enough that when 
> the admin for the manager 3 levels above me tries to resize the images 
> that I've cut and pasted, it doesn't choke (that means using WMF or EMF 
> for graphics, for instance).

Hmmm... I typically try to use a more standard image format (png, jpg, 
tiff, ...).  I have found these work better than others in my apps. 
Most everything in the windows world happily deals with those.  I have 
had lots of problem with WMF import.  Don't know why.

> What does this sort of environment mean?  It means that a strategy where 
> I run my analysis tool on a Linux box and then try to export the data 
> back to my Windows box for doing the reports is a royal pain.  It's 

Hmmm.... If this is a royal pain then (and please don't take this wrong) 
your linux/windows system is set up wrong.  To "export"/"import" I open 
a file on our SAMBA share, that happens to house the home directories 
from the cluster.   If you have to do anything else then something 
somewhere is broken, and yes, it is a pain.

We set up the transparent access for our customers, and I had assumed 
that all cluster vendors did.  Maybe I am wrong.

> worse than sneakernet.  Sure, I can SSH into the Linux box from my 
> windows box, and even fire up a Xserver on the Windows box, but things 
> like cutting and pasting just don't work very seamlessly, and it seems 
> that Linux application creators consider generating Windows compatible 
> file formats anathema (leaving aside the file format aspects..) because 
> they might be considered "pandering to the dark side".  Folks.. 
> uncompressed TIFF images don't hack it as an interchange medium.

What?  16MB TIFF for a 50k JPEG?  I usually use JPEG or png.  If I need 
to use a windows format I do.  Linux can read/write all of them (and 
interconvert) using the right tools, so this is rarely an issue.

As for "pandering", well, that may be an option for some, but my 
customers and users need to get work done.  The issue is how do you get 
it done in the least amount of time and effort, and the greatest impact.

> And no, Open Office is not fully interoperable with MS Office.  There's 
> always little hiccups with things that you really, really need.

charts (cough cough)


> (hmm.. 
> equation editor?  footnotes?  change tracking? Outline mode?)  The 

Actually all of these work really well in 2.0.  Its the charts that 
drive me batty.

> typical scenario is that you're one of half a dozen folks working on a 
> document, and you all pass it back and forth and make changes, and for 
> all practical purposes, we ALL have to be using the same tools (even 
> going back and forth between Mac and PC is problematic.. Those "big red 
> X" things that appear in your ppt slides).

Hmmm.  Been doing this without problem for a while for ppts going to 
certain groups that like good ppts.  For the most part we haven't run 
into problems apart from font mapping, and I am admittedly not 
interested in fixing it at this moment.

> 
> Let's be realistic.. as a hypothetical small user of a cluster for some 
> analytical task, more than 50% of my time is going to be spent not doing 
> the analysis, but in dealing with other aspects of the job: 
> administrative reporting; writing budgets; generating reports; creating 
> proposals for new work.  We leave aside here scenarios where I get to 
> manage a cadre of cluster monkeys who I get to tell "do this analysis, 
> produce this report, make it so", in which case I'm really not the 
> cluster user, but rather the analysis buyer.

True.

> 
> So, whatever applications I'm using on my cluster have to seamlessly 
> integrate with the tools the "rest of the business world" are using, 
> whether I like or not.

Also true, quite true of my customers.  They are looking for minimum 
time to insight (hey look, its a marketing phrase).  Fussing around with 
transport and conversion of data is not an option.  So they don't have to.

They can mount their home directories on the cluster as yet another 
drive letter.  They can and do use it as yet another resource.

> 
> Now, let's consider another practical detail..  I've got my cluster 
> running, and I'm cracking through my work.  Something breaks (maybe a PC 
> rolls over and dies).  I call the help desk.  The vast majority of 
> problems are something simple (whether the cluster is Linux or 
> Windows).  The IT organization has dozens of folks familiar with getting 
> Windows PCs fixed and running: after all, they've got all those 
> thousands of Windows desktops to support.  Probably any one of them can 
> come and swap out disk drives on my cluster nodes, or bring up the spare 
> node.  Say my IT support organization does actually support Linux too, 
> but, in view of the realities, Linux is probably less than 5% of the 
> installed base, so the support staff for Linux boxes is 1/20th of that 
> for Windows. If you have 10,000 installed Windows desktops, you probably 
> have around 50-100 support people for those desktops, of which perhaps 
> 10 are real crackerjack skilled ones who can take on the peculiarities 
> of your cluster.  You might have 5 who support Linux, and only 1 who 
> might know something about clusters.

Thats why there are support experts out there (ahem) selling their 
services.

> The odds of getting someone to fix my broken cluster, today or tomorrow, 
> are much higher if it's Windows based, just because there's more folks 
> around who are capable of doing it. If that 1 Linux cluster weenie 
> happens to be on vacation, I'm dead... the odds of all 10 Windows 
> cluster weenies being on vacation simultaneously is much lower.

Hmmmm.  Again, thats why there are external experts who do this stuff. 
As for the 50-100, I think the number is closer to 20-50 desktops per 
admin.  I have seen 4000 node clusters supported by 2 people full time. 
    I am not going to comment on the other aspects of this.

> Now let's talk security.  My speculative IT organization supports 10,000 
> windows desktops, and has fairly systematic and rigorous ways to deal 
> with the patches that come out once a month, as well as hotfixes for 
> vulnerabilities that get discovered.  My Windows based cluster isn't 
> going to seem scary to the IT security folks.. it's just another 100 
> computers and represents an infinitesimal increase in the overall 
> workload and a small increase in the complexity of their workload. The 
> incremental cost to bring my cluster into the corporate fold, from a 
> security standpoint, is small.

Uh.... I am going to disagree with you.  If you saw the firedrills these 
folks go through when the patch bolus hits... it breaks standard 
desktops and servers, and they need to watch everything very carefully. 
  I know of at least one fortune 500 IT organization that lives in fear 
and trepidation of the next patch bolus.   They are serviced by some 
other fortune 500 organization for most of their IT stuff.  I will not 
comment on the quality of that support.

Look carefully at what the CTC has to go through with their systems.  If 
you are running 1000 copies of Norton on your disks, with each one 
loaded up with a personal fire wall, anti spyware and virus ... do you 
really have a cluster?  I don't think so.

> Say I wanted to install a Linux cluster.  Ooops.. they're not quite as 
> familiar with that.  

And this is why there is a market for this expertise.

> They don't have all the patch rollout stuff, they 
> don't have a patch validation methodology, etc.  Sure, there's all kinds 
> of patch management stuff for Linux, in a bewildering variety of 
> options, but now we've got to have a Linux security expert, in addition 

... all of this outsourceable for a tiny fraction of what they pay for 
their required in house windows staff ...

> to the cadre of MS security folks we already have. You mean your cluster 
> uses a different distro than the other iconoclastic Linux desktop users 
> have? You recompiled the kernel to get the latest whizbang high 
> performance network support?

Hmmmm.....

> 
> With MS, the choice is easy.. use what you're already using for the rest 
> of the company (SMS probably).  Kernel or distro compatibility isn't an 
> issue.. you use what you're given and suck up the inefficiencies and 
> live with it.  If it's a performance dog, you go make the pitch to buy 
> more nodes.

Actually it is a huge issue.  Some codes are just not supported under 
some patch levels (SP2) because it breaks it.  So you have a choice of 
running your mission critical code or running at the accepted support 
level.

> Then there's the whole "hooking my box to the corporate network" 
> thing... Most corporate IT infrastructure folks get pretty picky about 
> what's hanging on the net, especially if you're using some sort of 
> tunnel to talk to it.  They might want you to put a third party firewall 
> between your cluster and the network, which all of a sudden not only 
> increases the cost, but also means that it might be hard for you to sit 
> at your desktop machine and talk to your cluster down the hall.

Haven't run into this yet.  Most want it tightly integrated into the 
net, with the only real pain being AD and other things for 
authentication.  That and crappy DNS systems.

> So, all in all, there's a real case to be made for a Windows based 
> cluster, even if the raw performance takes a big hit.  In terms of 
> "getting the work done" for a fixed dollar allocation, you might be 
> better off buying more nodes to make up for the performance than paying 
> for all the  extra stuff that corporate IT is going to require.

There is a case, but I think it is different than you argued.  I have 
been thinking this through for a while.  A large cluster is an 
appliance, as is a small cluster, as is a router.  It needs to be 
managed as one, and appear to drop onto the net as an appliance.  All 
interaction from the windows centric folk need to be as an appliance. 
  This is the windows model.

It is very hard to get a windows person to change to linux, unless they 
need to.  So you lower the barriers by making this thing appear as nice 
and friendly as possible.  Wrap it with some nice web tools  for job 
submission (we have some being deployed at a number of customer sites). 
  Make data transfer drag and drop via explorer (which we do).  Make all 
aspects of this be as easy as possible.

It can be done, just requires effort and focus upon integration.

There are lots of cluster hardware vendors out there who will happily 
sell linux hardware (or windows for a premium).  There are few who 
actually know how to make this all work seamlessly.   If you call it a 
cluster appliance and build it as such, and make it appear to be just a 
little web appliance with some disk which happens to calculate for 
people, well, the resistance level is very low.

Just my $0.02/.

> 
> 
> James Lux, P.E.
> Spacecraft Radio Frequency Subsystems Group
> Flight Communications Systems Section
> Jet Propulsion Laboratory, Mail Stop 161-213
> 4800 Oak Grove Drive
> Pasadena CA 91109
> tel: (818)354-2075
> fax: (818)393-6875
> 
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 734 786 8452
cell : +1 734 612 4615