[Beowulf] /etc/security/limits.conf and Torque jobs

[William G Wichser]

----- Original Message -----
From: Mark Hahn <hahn at physics.mcmaster.ca>
Date: Wednesday, May 17, 2006 6:51 pm
Subject: Re: [Beowulf] /etc/security/limits.conf and Torque jobs
To: Bill Wichser <bill at Princeton.EDU>
Cc: beowulf at beowulf.org

> > So I submit a job, through Torque, and I simply execute a "ulimit 
> -l" 
> > only to find that my limit is set to the original value, 32.  
> Using a 
> 
> inherited from the process that started your process, probably.
> try restarting relevant scheduler daemons - it matters when they
> started up relative to your limits.conf changes.
> 
> incidentally what leads you to think you should be using memory 
> locking?it's generally a very bad idea, since, like running in RT 
> scheduling 
> priorities, you're preventing the kernel's normal balancing...
> 

Yes, right.  Driving home it occured to me that I probably should restart that daemon.  Sure enough, changes are inherited.

Now as to why.  OpenMPI with OpenIB.  See thread:
http://www.open-mpi.org/community/lists/users/2005/12/0425.php

basically a: mca_btl_openib_module_init] error creating high priority cq for mthca0 errno says Cannot allocate memory
The recommended fix is to increase this value.  OpenMPI must be trying to nail down some memory for  optimizing some 
memory manuevers.  I'd have thought this would be pinned down in the OpenIB driver code though.

For now, I'm just trying to get it to work!

Bill