[Beowulf] Users abusing screen

[Guy Coates]

On 21/10/11 14:10, Prentice Bisbal wrote:
> Beowulfers,
> 
> I have a question that isn't directly related to clusters, but I suspect
> it's an issue many of you are dealing with are dealt with: users using
> the screen command to stay logged in on systems and running long jobs
> that they forget about. Have any of you experienced this, and how did
> you deal with it?
> 
> Here's my scenario:
> 
> In addition to my cluster, we have a bunch of "computer servers" where
> users can run the programs. These are "large" boxes with more cores
> (24-32 cores) and more RAM (128 - 256 GB, ECC) than they'd have on a
> desktop top.
> 
> Periodically, when I have to shutdown/reboot a system for maintenance,
> I find a LOT of shells being run through the screen command for users
> who aren't logged in. The majority are idle shells, but many are running
> jobs, that seem to be forgotten about. For example, I recently found
> some jobs running since July or August that were running under the
> account of someone who hasn't even been here for months!
> 
> My opinion is these these are shared resources, and if you aren't
> interactively using them, you should log out to free up resources for
> others. If you have a job that can be run non-interactively, you should
> submit it to the cluster.
> 
> Has anyone else here dealt with the problem?
> 
> I would like to remove screen from my environment entirely to prevent
> this. My fellow sysadmins here agree. I'm expecting massive backlash
> from the users.
> 
> 

On our interactive/login nodes we use the auto-nice deamon
(http://and.sourceforge.net/), which will nice/kill processes which have
used up excessive walltime/cputime, and fascist memory limits in
/etc/security/limits.conf which prevent people from running jobs that
gobble up memory.

We use wrappers around our most common "heavy" interactive commands (eg
R) which transparently submit them as interactive jobs to out farm
worker nodes.


Cheers,

Guy

-- 
Dr. Guy Coates, Informatics Systems Group
The Wellcome Trust Sanger Institute, Hinxton, Cambridge, CB10 1HH, UK
Tel: +44 (0)1223 834244 x 6925
Fax: +44 (0)1223 496802