[Beowulf] Re: distcc on Beowulf?

[Joel Krauska]

scunacc wrote:
> Yes, I've been doing that for some little while now.

Awesome.  Good to hear, Derek.
I'd love to hear your solution.


I got it running myself, and it's looking hopeful, but having some 
problems doing a kernel build.

Your comments about how Bproc and distcc are orthogonal was right on the 
mark..  distcc was having problems because the compute nodes didn't 
mount /usr. (Bproc doesn't really come in to the picture at all.)

By adding a /usr mount point to /etc/exports and /etc/beowulf/fstab I 
was able to get distcc running.


However as I hinted above, I seem to be running in to a kernel build 
problem using distcc.

make -f scripts/Makefile.build obj=arch/x86_64/ia32
   gcc -Wp,-MD,arch/x86_64/ia32/.syscall32.o.d -nostdinc -iwithprefix 
include -D__KERNEL__ -Iinclude  -Wall -Wstrict-prototypes -Wno-trigraphs 
-fno-strict-aliasing -fno-common -O2     -fomit-frame-pointer 
-mno-red-zone -mcmodel=kernel -pipe -fno-reorder-blocks 
-Wno-sign-compare -fno-asynchronous-unwind-tables 
-Wdeclaration-after-statement    -DKBUILD_BASENAME=syscall32 
-DKBUILD_MODNAME=syscall32 -c -o arch/x86_64/ia32/syscall32.o 
arch/x86_64/ia32/syscall32.c

{standard input}: Assembler messages:
{standard input}:5: Error: file not found: 
arch/x86_64/ia32/vsyscall-syscall.so
{standard input}:8: Error: file not found: 
arch/x86_64/ia32/vsyscall-sysenter.so
distcc[20537] ERROR: compile arch/x86_64/ia32/syscall32.c on .0 failed
make[1]: *** [arch/x86_64/ia32/syscall32.o] Error 1
make: *** [arch/x86_64/ia32] Error 2


These files exist and show up with a
bpsh -a ls -l arch/x86_64/ia32/


The above gcc command works perfectly when called locally on the master 
node, and fails when run under bpsh.

The best I can think is that perhaps something in the make or shell 
environment is missing?  (PATH?)

I realize x86-64 is relatively new territory, so I might downgrade my 
cluster to a 32-bit system, since this could be difficult to find others 
  who can reproduce, and I could figure out if it has something to do 
with building for this arch.

Again, any insight in to your methods would be appreciated.

Thanks,

--joel