Suggestion for gcc build farm

[Jakob Østergaard]

On Tue, Jul 24, 2001 at 07:54:04AM -0700, Ron Chen wrote:
> Why don't you use qmake from the Sun Grid Engine(SGE)
> package. SGE is a distributed resource package.

I needed this a year ago, and I couldn't find any usable solution back then.

It turns out, that yet another alternative exists:  jobd, which by total
coincidence was the name I chose for my project - I renamed my project to ANTS
later.  Perhaps I didn't look very carefully   :)

One of the main problems with the existing systems was their requirements (NFS
exported queue directories, MOSIX installations, PVM, etc. etc.) and the
*latency*.  Even a batch queuing system with a "now" queue has a significant
latency if you're distributing a few hundred compilations that each last 5-15
seconds.  My workload here is mixed, with some very short compilations, and
some very large ones.  I need low latency - I don't need an e-mail when a job
completes   :)

There are differences though:  ANTS is, as the name implies (autonomous
networked task scheduler) "autonomous".  It means, there is no central
scheduler.  You can take down any node in the cluster and the scheduling will
still work.  This is useful if you base your cluster solely on workstations.

jobd on the other hand seems to be more mature when it comes to status
reporting tools etc.

I planned to make a patch for GNU Make so that ANTS would be transparently
integrated with the -j option. This is probably something I will do later if noone
beats me to it.  For now you have to prefix your compile jobs with eg. "rant -t
g++" - but that's easily accomplished with makefile rules of course.  It's just
not "transparent".

I have no experience with the Sun product you mention.  What are the pro's and
con's   ?

> 
> You can have your requests queue up, if there are too
> many developers using the cluster. Also, you can run

Same with ANTS - naturally you need to limit the number of jobs on each
workstation.

> your test scripts using SGE. And the distributed make,
> qmake, can speed up the make process.

Instead of typing "rsh somehost gcc" you can type "rant -t gcc gcc", and ANTS
will choose the best host for a gcc job and execute it on that host.  Think of
"rant" as a "smart rsh".  You give it a job type, and it will pick the best
host for you.   This is how ANTS work, today.

If someone (hint hint  ;)  patched GNU Make (which already has hooks in place
for this kind of distribution tools) - you would have a parallel *standard* GNU
Make.  I previously tried nmake which has some parallel hacks in it (or there
are patched versions that have at least), but I need GNU Make.

I don't know if qmake is sufficiently GNU Make compatible to suit my needs.
With ANTS you're not dependent on any make utility.  If you can use rsh, you
can use rant.

Maybe your situation is different, but to me GNU Make is the necessary standard
and I don't see that changing anytime soon.   If a distribution tool requires me
to change my make utility it's most likely not an option for me.

> 
> SGE is free+opensource. 

Nice.

Will it stand up to an attack on the network (eg. a remote attacker sending
spoofed packets)  ?

It was my impression at the time that many of the more established solutions
(for example, anything involving PVM) were inherently vulnerable to network
attacks and *had* to be put on a protected network.

ANTS should stand up well to remote attacks, I'm fairly confident that it has
no buffer-overflow holes, and it cannot be used as a DDoS amplifier.

ANTS will however be vulnerable if the attacker has control over the local area
network (if the attacker can intercept and inject packets as he pleases).
However, ANTS will refuse to run jobs as root, so the best an attacker with
control over the network can do, is to run jobs as ordinary users.   I thought
this was a fair tradeoff - after all there's no cryptographic code in ANTS at
all.

> 
> Synopsys use SGE and they saved lots of $$$.
> 
> http://www.sun.com/presskits/workstations/dac2001/Synopsys.pdf


I don't know how ANTS or jobd compare to the product you mention, but it is
pretty obvious that distributed compilation is a very good idea for large 
projects     :)

-- 
................................................................
:   jakob at unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............: