[Beowulf] Understanding environments and libraries caching on a beowulf cluster

[Jörg Saßmannshausen]

Dear all,

what we are doing is we are exporting our software stack via a shared file 
system like, for example NFS (not a good idea for larger clusters), 
SpectrumScale (formerly known as GPFS), Lustre, Ceph... the list is long.

To ease your pain with building architecture specific software and all the 
dependencies, I highly recommend EasyBuild for that:

https://easybuild.io/

It is a bit of a learning curve but then what is not. The big advantage here 
is there are currently over 2700 *individual* packages supported, not 
including extensions. Also, if you got a new cluster and you want to say 
install GROMACS from scratch, you can do it like this:

$ eb --rd GROMACS-2021.5-foss-2021b.eb

and that version is installed. You want CUDA support? Here it is:
$ eb --rd GROMACS-2021.5-foss-2021b-CUDA-11.4.1.eb

The community is constantly adding new software packages to it, so with each 
release there will be more. 

Also, unless you are already doing so, I recommend the use of Lmod instead of 
EnvironmentModules. 

I hope that helps a bit regarding taking care of libraries. 

All the best from a overcast and a bit windy London

Jörg 

Am Dienstag, 28. Juni 2022, 10:32:17 BST schrieb leo camilo:
> # Background
> 
> So, I am building this small beowulf cluster for my department. I have it
> running on ubuntu servers, a front node and at the moment 7 x 16 core
> nodes. I have installed SLURM as the scheduler and I have been
> procrastinating to setup environment modules.
> 
> In any case, I ran in this particular scenario where I was trying to
> schedule a few jobs in slurm, but for some reason slurm would not find this
> library (libgsl). But it was in fact installed in the frontnode, I checked
> the path with ldd and I even exported the LD_LIBRARY_PATH .
> 
> Oddly, if I ran the application directly in the frontnode, it would work
> fine.,
> 
> Though it occured to me that the computational nodes might not have this
> library and surely once I installed this library in the nodes the problem
> went away.
> 
> # Question:
> 
> So here is the question, is there a way to cache the frontnode's libraries
> and environment onto the computational nodes when a slurm job is created?
> 
> Will environment modules do that? If so, how?
> 
> Thanks in advance,
> 
> Cheers