[Beowulf] perl with OpenMPI gotcha?

David Mathog mathog at caltech.edu
Mon Nov 23 19:32:31 UTC 2020



Sounds like MPI isn't supported very well now, and perhaps it never was.

In each application the remote jobs all run the same script but they 
receive slightly different parameters which are read from the
command line.  That was easy with PVM since each remote 
job start had its own parameter list.  Using pdsh those parameters would, 
it seems, have to go in through node/job specific configuration files 
which the scripts would need to be modified to read.  (Or possibly through 
environmental variables, but it would still need more modification of the
compute node scripts.) So probably the quickest port will be 
to just use Parallel::Forkmanager and in the child processes run
ssh jobs using the existing script parameter lists.  That way there will
still be pids to work with and the master can still loop to keep track of 
which ones are running or have completed.  The jobs operate only on files 
on an NFS file system (for input, output, and status) and compute node 
local (scratch files, if any).  No data is fed in through stdin or returned
through stdout.

Thanks for your help,

David Mathog


More information about the Beowulf mailing list