[Beowulf] Lmod question

[Jörg Saßmannshausen]

Hi David,

that looks like a caching problem to me. I came across some oddities with Lmod 
before and most of the time regenerating the cache helps. 

If I recall correctly, there is a parameter for how long the cache is valid. 
So you either set it very low, and the cache will be regenerated very often, 
or very high, so the cache will be generated once and stays then like this.

Maybe that is something you want to look into?

I don't have access to a machine using Lmod right now else I would dig for the 
name of the cache-timeout. 

All the best from a dark London

Jörg


Am Freitag, 19. Juni 2020, 00:18:53 BST schrieb David Mathog:
> Calling all Lmod gurus...
> 
> I have been migrating packages from one machine to another, adding their
> Lmod lua files as I go using this script:
> 
> https://saf.bio.caltech.edu/pub/software/linux_or_unix_tools/module_generate
> _from_directory.sh
> 
> This is done from an account "modules" and normally as soon as that script
> has completed doing this:
> 
> module avail 2>&1 | grep newpackagename
> 
> will find the new package.  For some reason today that stopped working,
> or at least working immediately for the modules account.  But it still
> worked immediately for root.  File protections were all good, Selinux was
> ruled out.  After poking around this appears to be related to
> cache files, because when "modules" does:
> 
> module --ignore_cache avail 2>&1 | grep newpackagename
> 
> it sees the new entry.   The cache files were indeed located:
> 
> #as modules
> ls -al ~/.lmod.d/.cache
> total 424
> drwxrwxr-x. 2 modules modules     80 Jun 18 14:37 .
> drwxrwxr-x. 3 modules modules     20 Nov  1  2019 ..
> -rw-rw-r--. 1 modules modules 200725 Jun 18 10:55
> spiderT.x86_64_Linux.luac_5.3
> 
> and 10:55 is about the time things started to go wrong.  On my other
> machines "~/.lmod.d" does not exist.  On this machine root has these files
> too, in fact two of them,  and it seems to have updated recently:
> 
> -rw-r--r--. 1 root root 269155 Jun 18 14:17 spiderT.x86_64_Linux.lua
> -rw-r--r--. 1 root root 204618 Jun 18 14:17 spiderT.x86_64_Linux.luac_5.3
> 
> On the problem machine and the ones which do not have this problem the
> cache configuration is the same:
> 
> module --config 2>&1 | grep -i cache
> number of cache dirs               0
> Ignore Cache                       no
> Cached loads                       no
> User cache valid time(sec)         86400
> Write cache after (sec)            2
> 
> Another oddity - if a new "modules" session is started on the problem
> machine, suddenly the missing modules are listed in
> "module avail", not only in that old session, but also in the new one.
> But the time stamps on the .cache files do not change.
> 
> I have not done anything explicitly to generate those cache files.  The
> only thing modified in the files Lmod installed where the paths in
> 
> /etc/profile.d/00-modulepath.sh (and .csh)
> 
> The output of "module --config" is the same on the bad machine as on the
> good ones (other than things like differences in some version numbers,
> TCL's for instance.)
> 
> Any idea what might be going on here?  My best guess is that
> some side effect is running the spider process and generated those
> cache files, and then "module" checked them, even though it was
> configured not to (I think, see above.)  Ideally I would like to prevent
> it from making those cache files again, kind of hard to do not knowing what
> made them in the first place!
> 
> This is on CentOS 8, Lmod 8.2.7-1.
> 
> Thanks,
> 
> David Mathog
> 
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> https://beowulf.org/cgi-bin/mailman/listinfo/beowulf