[Beowulf] Win64 Clusters!!!!!!!!!!!!

Tue Apr 3 19:32:25 PDT 2007

The job scheduler is responsible for managing the user credentials and
tokens as well as allocating CPUs to jobs. So if you are not going to
use our scheduler, you need a different solution for the credential
management. How do the user's processes get created with the right
token? MPICH2 has a couple different solutions with different tradeoffs
- I'm not sure which one you use.

We have had requests for two different scenarios - enabling other MPI
stacks to use the credential management of our job scheduler and
enabling other job schedulers to do the credential management for our
MPI stack. We are looking at both of these for the next release but they
didn't make V1.

So like most product decisions, we didn't "have to do this" it was just
the normal tradeoff you make between features and schedules with a
healthy dose of security paranoia thrown into the mix.

John Vert
Development Manager
High Performance Computing

-----Original Message-----
From: Bill Bryce [mailto:bill at platform.com] 
Sent: Tuesday, April 03, 2007 1:12 PM
To: John Vert; Robert G. Brown
Cc: beowulf at beowulf.org
Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!!

Can I ask one simple question.....

Why did Microsoft have to do this (and I quote from MSDN)

"Every job and task that uses Microsoft MPI must be submitted through
the Microsoft job scheduler. There are several ways to submit jobs and
tasks to the job scheduler:"

I do not have that restriction with other MPI's such as MPICH2 for
Windows from Argonne.

Regards, 

Bill.

-----Original Message-----
From: John Vert [mailto:jvert at windows.microsoft.com] 
Sent: Tuesday, April 03, 2007 3:19 PM
To: Robert G. Brown; Bill Bryce
Cc: beowulf at beowulf.org
Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!!

I'd like to clear up a few misconceptions on this thread.

1. There is no "MS specific library that abuses 'standard MPI'". The #1
goal for our MPI implementation is to make it as simple as possible for
software vendors to port their codes to Windows and our MPI stack. That
is why we based our implementation on MPICH2. The feedback we have
gotten from software vendors has shown this approach was successful and
we have had very few issues with MPI incompatibilities. We have not
added anything outside the MPI spec.
2. MPICH2 is not licensed under GPL or under BSD. See
http://www-unix.mcs.anl.gov/mpi/mpich2/license.htm  Many commercial
closed-source MPI implementations have been based on MPICH. Microsoft is
actively working with Argonne on incorporating the changes we made back
into the MPICH2 distribution. Argonne has been great to work with and
really supportive of our efforts.
3. The changes we made can be loosely grouped into three areas.
Performance, security, and scheduler integration. We handle the user
credentials differently than Argonne's Windows implementation which is
different than Argonne's Linux implementation due to some fundamental
differences between the two operating systems. The scheduler integration
is pretty straightforward - things like cleaning up all MPI child
processes when a job is cancelled, and automatically propagating things
like the allocated nodes and the number of CPUs from the scheduler to
mpiexec. I don't think this is significantly different than what other
job schedulers have done.
4. If you want to learn more about Windows HPC clusters, I recommend
checking out www.microsoft.com/hpc and www.windowshpc.net. I'm also
happy to answer questions on this list, but frankly the S/N ratio tends
to drop dramatically as soon as someone mentions Windows or Microsoft.

John Vert
Development Manager
High Performance Computing

-----Original Message-----
From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org]
On Behalf Of Robert G. Brown
Sent: Tuesday, April 03, 2007 7:59 AM
To: Bill Bryce
Cc: beowulf at beowulf.org
Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!!

On Tue, 3 Apr 2007, Bill Bryce wrote:

> So this change effectively ties Microsoft MPI to only Microsoft
Windows
> platforms, and the security changes are closed source.  Not all of
> Microsoft's partners like MS MPI - when HP ships Microsoft CCS they
> remove MS MPI and put in HP MPI - which probably just adds to
confusion
> on the end user side.

Security changes?  To MPI itself?  This puts them into a somewhat grey
area with regard to the GPL, doesn't it... that viral thing.

I'm sure that they can manage the remote job launch any way that they
like, but doesn't that leave the MPI CODE still portable?

It isn't completely crazy that the US government would intervene here if
they broke MPI portability.  After all, MPI exists at all primarily
because of direct government intervention (unlike PVM, which exists
because some Very Bright People conceived it and invented it and made it
fly).

Still, nothing the Borg Empire does in this regard would surprise me.

     rgb

>
> Regards,
>
> Bill.
>
>
> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org]
> On Behalf Of Robert G. Brown
> Sent: Monday, April 02, 2007 8:22 PM
> To: Tom Mitchell
> Cc: beowulf at beowulf.org
> Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!!
>
> On Mon, 2 Apr 2007, Tom Mitchell wrote:
>
>> On Sun, Apr 01, 2007 at 02:02:53PM +0500, amjad ali wrote:
>>> Date: Sun, 1 Apr 2007 14:02:53 +0500
>>> From: "amjad ali" <amjad11 at gmail.com>
>>> To: beowulf at beowulf.org
>>> Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!!
>>>
>>> Hi All,
>>>
>>> Would any of you please like to share
usage-experience/views/comments
>>> about Windows Compute Cluster Server 2003 based Beowulf Clusters?
>>>
>>> What in your opinion is the future of such clusters?
>>>
>>> How you compare these with the LINUX CLUSTERS?
>>
>> With full consideration to "fat, soft pretzels with
>> cheezy-mustard sauce or rolled in asiago parmesan and garlic."
>>
>> MS pulled a version of mpich/mvapich/MPI and ported it to windows.
>> They also developed some library code to gateway some *inx
> library/system
>> calls to windows.  The root sources of MPI are public and not GPL so
> they can.
>>
>> It might be worth looking at the MS announcement -- but why
>> bother.  If you look you might think that common MPI codes
>> would just compile and run...  I have no idea I expect some will
>> and there begins silly porting for the next...
>
> Sure.  MS did this, no doubt.  And as you note below, no sooner do
they
> get it in when they begin the borgification of MPI, just as they've
> borgified java, c, c++, and anything in the Universe they can sucker
> somebody into buying in borgified form.
>
> Borgifying MPI is the most humorous thing in the Universe, BTW, given
> its historical origins -- it was basically a language written
> (reluctantly!) by supercomputer vendors when the US government got
tired
> of paying for all their important codes to be ported to each new
> generation of proprietary hardware with its proprietary low level
calls.
> MS is doubtless trying to figure out just how much of that they can
> undo while building up a big enough market share and enough vendors of
> closed source applications written with their borgisms that they
can...
>
> Oh wait.  It IS GPL.  Do you think that they actually read it?
>
> However, I was really referring to the other aspects of program
> development and performance tuning associated with using a closed
source
> development environment.
>
> Resistance is Futile.
>
>     rgb
>
>>
>> Once a set of boxes are interconnected and you have library
>> support for MPI or another way to share data (PVM... whatever)
>> you are off and running in the clustering world.   Sadly MS
>> has a MS specific library that abuses "standard MPI" and could
>> quickly cause source code to surface that runs correctly or on a
>> MS cluster but not on another OS based cluster (Linux, Solaris,
>> Irix, AIX).   I see this all the time with java script, and c,
>> c++, and other codes where little 'features' hook you in.
>>
>> Some will be fooled into thinking that this is something to look at
>> or worse something to spend money on.
>>
>> SUMMARY:
>> Since you posted this on 1 Apr 2007 all I can do is giggle
>> and wonder why I replied.
>>
>> Regards,
>> mitch
>>
>> PS: Ask in a year but not on April fools/joke day.
>>
>>
>
>

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf