[Beowulf] Understanding environments and libraries caching on a beowulf cluster

[Jonathan Aquilina]

Hi Leo,

A bit more clarification on this cluster setup

You have 1 primary node and then the rest connect to it in a diskless fashion booting off the primary (front) node?

Regards,
Jonathan
________________________________
From: Beowulf <beowulf-bounces at beowulf.org> on behalf of leo camilo <lhcamilo at gmail.com>
Sent: 28 June 2022 11:32
To: Beowulf at beowulf.org <Beowulf at beowulf.org>
Subject: [Beowulf] Understanding environments and libraries caching on a beowulf cluster

# Background

So, I am building this small beowulf cluster for my department. I have it running on ubuntu servers, a front node and at the moment 7 x 16 core nodes. I have installed SLURM as the scheduler and I have been procrastinating to setup environment modules.

In any case, I ran in this particular scenario where I was trying to schedule a few jobs in slurm, but for some reason slurm would not find this library (libgsl). But it was in fact installed in the frontnode, I checked the path with ldd and I even exported the LD_LIBRARY_PATH .

Oddly, if I ran the application directly in the frontnode, it would work fine.,

Though it occured to me that the computational nodes might not have this library and surely once I installed this library in the nodes the problem went away.

# Question:

So here is the question, is there a way to cache the frontnode's libraries and environment onto the computational nodes when a slurm job is created?

Will environment modules do that? If so, how?

Thanks in advance,

Cheers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://beowulf.org/pipermail/beowulf/attachments/20220628/6d479466/attachment.htm>