Image Processing on a BeoWulf

[Mark Lucas]

Looking for some guidance on our strategy for satellite and aerial 
image processing on a BeoWulf cluster.  We have successfully added 
PVM into our production software.  Essentially, very large image 
processing jobs are sectioned into image tiles and the tiles are 
farmed out to the nodes for the mathematical transformations (image 
processing functions, image space to world space to map projection 
space transformations).  On individual jobs this works very well as 
the process is extremely CPU bound. As a result we are seeing close 
to linear scaling on individual jobs.  For each job we use all of the 
available processors in the cluster.

Our desire is to allow many jobs to be initiated ad-hoc by the 
production operators or by web based clients and take advantage of 
the parallelism and scaling offered by the cluster.  Whenever we have 
attempted that we seem to lock the system up.   We see the same 
symptoms when we inadvertently schedule a single job and ask for more 
processors than are currently available.  We currently have a work 
around in place - we implemented a queue that lines up the submitted 
jobs sequentially.  This certainly is not optimal as small jobs have 
to wait behind large ones.

We are learning as we go and are looking for advice and guidance.  is 
this expected behavior from PVM?  Will we need to look at MPI?

Any help and advice would be greatly appreciated.

BTW, We will be in Atlanta showing this off (hopefully on more than 
one job at a time :-) )

Thanks,
Mark
-- 
**********************
Mark R Lucas
Chief Technical Officer
ImageLinks Inc.
4450 W Eau Gallie Blvd
Suite 164
Melbourne Fl 32934

321 253 0011 (work)
321 253 5559 (fax)

mlucas at imagelinks.com
**********************