[Beowulf] Understanding environments and libraries caching on a beowulf cluster

[leo camilo]

I see, so if I understand it correctly I have to make sure that there is a
copy of the library, environments and modules on every computational node?

I am wondering if I can get around it by using nfs.

On Tue, 28 Jun 2022 at 11:42, Richard <ejb at trick-1.net> wrote:

> For what it’s worth I use an easy8 licensed bright cluster (now part of
> NVidia) and I continually find I need to make sure the module packages,
> environment variables etc are installed/set in the images that are deployed
> to the nodes
>
> Bright supports slurm, k8, jupyter and a lot more
>
> Richard
>
> Sent from my iPhone
>
> > On 28 Jun 2022, at 19:32, leo camilo <lhcamilo at gmail.com> wrote:
> >
> > 
> > # Background
> >
> > So, I am building this small beowulf cluster for my department. I have
> it running on ubuntu servers, a front node and at the moment 7 x 16 core
> nodes. I have installed SLURM as the scheduler and I have been
> procrastinating to setup environment modules.
> >
> > In any case, I ran in this particular scenario where I was trying to
> schedule a few jobs in slurm, but for some reason slurm would not find this
> library (libgsl). But it was in fact installed in the frontnode, I checked
> the path with ldd and I even exported the LD_LIBRARY_PATH .
> >
> > Oddly, if I ran the application directly in the frontnode, it would work
> fine.,
> >
> > Though it occured to me that the computational nodes might not have this
> library and surely once I installed this library in the nodes the problem
> went away.
> >
> > # Question:
> >
> > So here is the question, is there a way to cache the frontnode's
> libraries and environment onto the computational nodes when a slurm job is
> created?
> >
> > Will environment modules do that? If so, how?
> >
> > Thanks in advance,
> >
> > Cheers
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> > To change your subscription (digest mode or unsubscribe) visit
> https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://beowulf.org/pipermail/beowulf/attachments/20220628/d4f89576/attachment-0001.htm>