[Beowulf] MPICH on heterogeneous (i386 + x86_64) cluster

[Ashley Pittman]

On Fri, 2005-01-21 at 09:41 -0500, Sean Dilda wrote:
> John Lau wrote:
> > Hi,
> > 
> > Have anyone try running MPI programs with MPICH on heterogeneous cluster
> > with both i386 and x86_64 machines? Can I use a i386 binary on the i386
> > machines while use a x86_64 binary on the x86_64 machines for the same
> > MPI program? I thought they can communicate before but it seems that I
> > was wrong because I got error in the testing.
> > 
> > Have anyone try that before?
> 
> I've not tried it, but I can think of a few good reasons why you'd want 
> to avoid it.  Lets say you want to send some data that's stored in a 
> long from the x86_64 box to the x86 box.  Well, on the x86_64 box, a 
> long takes up 8 bytes.  But on the x86 box, it only takes 4 bytes.  So, 
> chances are some Bad Stuff(tm) is going to happen if you try to span an 
> MPI program across architectures like that.

I've done it with MPI across ia32 and ia64 machines, purely as a
demonstration though, it's a headache to get it right and hard to see
why you'd want to except in very special circumstances.

As to whether MPICH can do this though it another matter (I don't know
the answer to this) but I really think you should try and find another
solution.

Running a ia32 binary on mixed ia64 and ia32 machines works much better
however, I've even done this between ia64 and x86_64, just keep
everything 32bit and it should all work.

I do know of people however who run a parallel application across alphas
and nVidia graphics cards plugged into p4's but this isn't using MPI.

Ashley,