[Beowulf] Understanding environments and libraries caching on a beowulf cluster

[Richard]

For what it’s worth I use an easy8 licensed bright cluster (now part of NVidia) and I continually find I need to make sure the module packages, environment variables etc are installed/set in the images that are deployed to the nodes

Bright supports slurm, k8, jupyter and a lot more

Richard 

Sent from my iPhone

> On 28 Jun 2022, at 19:32, leo camilo <lhcamilo at gmail.com> wrote:
> 
> 
> # Background
> 
> So, I am building this small beowulf cluster for my department. I have it running on ubuntu servers, a front node and at the moment 7 x 16 core nodes. I have installed SLURM as the scheduler and I have been procrastinating to setup environment modules.
> 
> In any case, I ran in this particular scenario where I was trying to schedule a few jobs in slurm, but for some reason slurm would not find this library (libgsl). But it was in fact installed in the frontnode, I checked the path with ldd and I even exported the LD_LIBRARY_PATH .
> 
> Oddly, if I ran the application directly in the frontnode, it would work fine.,
> 
> Though it occured to me that the computational nodes might not have this library and surely once I installed this library in the nodes the problem went away.
> 
> # Question:
> 
> So here is the question, is there a way to cache the frontnode's libraries and environment onto the computational nodes when a slurm job is created?
> 
> Will environment modules do that? If so, how?
> 
> Thanks in advance,
> 
> Cheers
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf