make and cluster software

[Schilling, Richard]

You might try the clusterit toolkit.  It allows you to spawn jobs across
several machines quite easily.  You can find it on the FreeBSD web site
(www.freebsd.org).


Richard Schilling
Web Integration Programmer/Webmaster
phone: 360.856.7129
fax: 360.856.7166
URL: http://www.affiliatedhealth.org

Affiliated Health Services
Information Systems
1971 Highway 20
Mount Vernon, WA USA


> -----Original Message-----
> From: Chris Black [mailto:cblack at eragen.com]
> Sent: Wednesday, May 30, 2001 11:41 AM
> To: beowulf at beowulf.org
> Subject: make and cluster software
> 
> 
> We are trying to move our large sets of batch jobs (several thousand)
> into a make type environment. We already have a cluster running 
> OpenPBS, but are looking at alternatives to PBS since we have been 
> having problems with hangs and such. I am interested in MOSIX since 
> it seems like it would integrate well with our multiple independant 
> processes model and with make. But I also read in the MOSIX FAQ that 
> java processes don't get migrated. Most of our programs are java 
> code that links in fast native code for algorithms. I am not that 
> familiar with MOSIX and I don't know if migrating processes would 
> even be necessary for us. Ideally I would like to be able to do a:
> make -j50
> 
> and run 50 processes at a time. Right now we just have make execute 
> a bunch of qsubs to queue jobs into PBS.
> We are also looking at Codine/GRD engine.
> 
> So, does anyone have any suggestions? Any experience with MOSIX and 
> make? Would this need the ability to migrate processes? Is MOSIX not 
> a good solution for this type of application? Anyone else setting up 
> make to use a cluster semi-transparently?
> 
> Thanks,
> Chris
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.scyld.com/pipermail/beowulf/attachments/20010530/072f97a5/attachment.html