[Beowulf] Re: distcc on Beowulf?

[scunacc]

Dear Joel,

> Awesome.  Good to hear, Derek.
> I'd love to hear your solution.

Sounds like you are doing the right thing anyway re: the mounting now. 

Not sure I could offer more unless you run into other issues.

You also want to make sure you are sharing the complete set of libraries
you need via the config file libraries section but that's about it.

How are you starting your remote distccd's?

I found that it was also useful to create /root/.distcc on each node.

> I got it running myself, and it's looking hopeful, but having some 
> problems doing a kernel build.
> 
> Your comments about how Bproc and distcc are orthogonal was right on the 
> mark..  distcc was having problems because the compute nodes didn't 
> mount /usr. (Bproc doesn't really come in to the picture at all.)
> 
> By adding a /usr mount point to /etc/exports and /etc/beowulf/fstab I 
> was able to get distcc running.

As I said above... ;-)

> However as I hinted above, I seem to be running in to a kernel build 
> problem using distcc.

>...
> {standard input}: Assembler messages:
> {standard input}:5: Error: file not found: 
> arch/x86_64/ia32/vsyscall-syscall.so
> ...
> These files exist and show up with a
> bpsh -a ls -l arch/x86_64/ia32/
> 
> ...
> The best I can think is that perhaps something in the make or shell 
> environment is missing?  (PATH?)

One thing to check: Where are the intermediate files being created? Is
/tmp accessible on all the nodes? Is that being shared (shouldn't be -
should be local to each node)? 

Also, - are the target dirs being mounted r/w on the nodes? If not, then
it might be having problems creating the target files. I take it you're
doing the kernel build in /usr/src/linux? Is /usr mounted ro or r/w
remotely?

Just a thought.

Kind regards

Derek Jones.