[Beowulf] Gridengine and bash + Modules

[Reuti]

Hi,

Am 18.01.2010 um 20:53 schrieb madskaddie at gmail.com:

> 2010/1/18 Brian Smith <brs at usf.edu>:
>> Ah, the RedHat-isms that we take for granted... hah!  I forgot  
>> that the
>> default ~/.bashrc I push out to everyone sources /etc/bashrc by  
>> default.
>> What distro are you using?
>>
>
> Debian lenny
>
>> There's also this bit of goodness from the man page:
>>
>> "Bash attempts to determine when it is being run with its standard  
>> input
>> connected  to a a network connection, as if by the remote shell  
>> daemon,
>> usually rshd, or the secure shell daemon sshd.
>>
>
> The Debian bash man page doesn't say the word "sshd" (only "rshd"),
> and I'm using ssh as the remote shell, so it may be the case (weird,
> but possible).
>
> (...)

a) you could define a starter method in SGE's queue setup to define  
the necessary things.

But what functions are these in detail? I checked on one cluster and  
to me it looks like it's only one:

module ()
{
     eval `/mypath/environment-modules/3.2.6//Modules/$MODULE_VERSION/ 
bin/modulecmd bash $*`
}

and the eval isn't necessary IMO. It would be necessary if $* would  
include something which has to be interpreted again. So an alias is  
also working for me (unset module and define it like below):

b) alias module='/cm/local/apps/environment-modules/3.2.6//Modules/ 
$MODULE_VERSION/bin/modulecmd bash $*'

or c) you could define a wrapper with a script and put it in /user/ 
local/bin or alike.

-- Reuti


>>
>> I wonder if sge_shepherd doesn't, in fact, trick shells into beha
>> ving
>> this way... I know I'm not using BASH_ENV and my modules environment
>> works correctly.
>>
>
> Just to be sure we aren't missing something: you can load a module
> inside the submit job, correct?
>
> Case 1:
>
>  - module load something
>  - qsub job.sh
>  - cat job.sh
> #!/bin/bash
> #(sge config stuff)
>
> mpirun ...
>
> #EOF
>
> Case 2 (what I pretend):
>  - qsub job.sh
>  - cat job.sh
> #!/bin/bash
> #(sge config stuff)
>
> module add something
> mpirun ...
>
> #EOF
>
>
>
>
>>
>> -Brian
>>
>> --
>> Brian Smith
>> Senior Systems Administrator
>> IT Research Computing, University of South Florida
>> 4202 E. Fowler Ave. ENB308
>> Office Phone: +1 813 974-1467
>> Organization URL: http://rc.usf.edu
>>
>>
>> On Mon, 2010-01-18 at 14:38 +0000, madskaddie at gmail.com wrote:
>>> 2010/1/16 Brian Smith <brs at usf.edu>:
>>>> I'm using this in our environment.  I've simply added the Modules
>>>> environment code to /etc/bashrc and /etc/csh.cshrc on all nodes  
>>>> (I use
>>>> puppet to manage everything, so this is easy).  This ensures that
>>>> Modules is properly integrated with your environment regardless of
>>>> whether you are using an interactive or non-interactive  
>>>> invocation of
>>>> these shells.  This works for SGE (I'm on 6.2u4, ATM)
>>>>
>>>
>>> But it seems that gridengine spawns like "bash script_name" so no rc
>>> files are read. Reading bash manpage, I found the BASH_ENV  
>>> environment
>>> variable:
>>>
>>> """
>>> When  bash  is  started non-interactively, to run a shell script,  
>>> for
>>> example, it looks for the variable BASH_ENV in the environment,
>>> expands its value if it appears there, and uses the expanded  
>>> value as
>>> the name of a file to read and execute.  Bash behaves as if the
>>> following command were executed:
>>>               if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
>>> but the value of the PATH variable is not used to search for the  
>>> file name.
>>> """
>>> (bash manpage)
>>>
>>> Right now I'm setting this variable and with the "-V" job submission
>>> flag it's working well (it does not work correctly without it)
>>>
>>> Gil
>>>
>>>
>>
>>
>
>
>
> -- 
> "
> It can't continue forever. The nature of exponentials is that you push
> them out and eventually disaster happens.
> "
> Gordon Moore  (Intel co-founder and author of the Moore's law)
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf