[Beowulf] Win64 Clusters!!!!!!!!!!!!

[Ashley Pittman]

Bill Bryce wrote:
> Regarding the 'borgification' of MPI on Windows....there is an opening
> for them to do this.  MPI did not define how you start the tasks when
> you launch a parallel job....it left that up to the MPI implementation.
> Now most MPIs use something reasonable, like say ssh - or starting a mpd
> ring to launch the tasks (and this is all open source), however I
> believe that for Microsoft MPI they wrote their own replacement for
> 'ssh' and also modified how the tasks are launched to ensure their MPI
> fits into Microsoft's security model, here is specific information on
> this from Microsoft's site

You'll find this is common to all MPI's.  What microsoft have done here
is no different.  I'd argue that ssh is also not reasonable in this day
and age so however MS have done this it doesn't need to be very good to
be on a par with some Linux setups.

> "An important improvement that MS MPI brings to MPICH2 is in how
> security is managed during execution of MPI jobs. Each job is run with
> the credentials of the user. Credentials are only present while the job
> is being executed on the compute nodes and are erased when the job
> completes. Individual processes only have access to their logon token,
> and do not have access to the credentials (user name and password) that
> generated the token or to the credentials used by other processes."

This is an implementation detail and on Linux not a very important one,
I think the fact it's quoted here says more about expectations of MS
security than anything else.  It's my impression that
security/credentials hasn't been a problem for most people in practice
so although there are ways of doing it they aren't widely used.

> So this change effectively ties Microsoft MPI to only Microsoft Windows
> platforms, and the security changes are closed source.  Not all of
> Microsoft's partners like MS MPI - when HP ships Microsoft CCS they
> remove MS MPI and put in HP MPI - which probably just adds to confusion
> on the end user side.

This is also common to all closed source MPI's (Including those based on
MPICH) and actually in practice common to most open source MPI's as
well, it's the age old ABI issue.

HP ships their MPI on *all* systems they sell, buy a Quadrics system
from HP and you get HP-MPI by default.  I don't think it's because their
MPI is necessarily any better than our MPI on the particular platform,
it's just that it's the HP-MPI and it's a HP platform.

Ashley,