[Beowulf] Contents of Compute Nodes Images vs. Login Node Images

[Prentice Bisbal]

Ryan,

When I was at IAS, I pared down what was on the compute nodes 
tremendously. I went through the comps.xml file practically line-by-line 
and reduced the number of packages installed on the compute nodes to 
only about 500 RPMs. I can't remember all the details, but I remember 
omitting the following groups of packages:

1. Anything related to desktop environments, graphics, etc.
2. -devel packages
3. Any RPMS for wireless or bluetooth support.
4. Any kind of service that wasn't strictly needed by the compute nodes.

In this case, the user's desktops mounted the same home and project 
directories and shared application directory (/usr/local), so the user's 
had all the the GUI, post-processing, and devel packages they needed  
right on their desktop, so the cluster was used purely for running 
non-interactive batch jobs. In fact, there was no way for a user to even 
get an interactive session on the cluster.  IAS  was a small environment 
where I had complete control over the desktops and the cluster, so I was 
able to this. I would do it all again just like that, given as similar 
environment.

I'm currently managing a cluster with PU, and PU only puts the -devel 
packages, etc. on the the login nodes so users can compile there apps 
there.

So yes, this is still being done.

There are definitely benefits to providing specialized packages lists 
like this:

1. On the IAS cluster, a kickstart installation, including configuration 
with the post-install script, was very quick - I think it was 5 minutes 
at most.
2. You generally want as few services running on your compute nodes as 
possible. The easiest way to keep services from running on your cluster 
nodes is to not install those services in the first place.
3. Less software installed = smaller attack surface for security exploits.

Does this mean you are moving away from Warewulf, or are you creating 
different Warewulf images for login vs. compute nodes?


Prentice

On 10/23/2018 12:15 PM, Ryan Novosielski wrote:
> Hi there,
>
> I realize this may not apply to all cluster setups, but I’m curious what other sites do with regard to software (specifically distribution packages, not a shared software tree that might be remote mounted) for their login nodes vs. their compute nodes. From what I knew/conventional wisdom, sites generally place pared down node images on compute nodes, only containing the runtime. I’m curious to see if that’s still true, or if there are people doing something else entirely, etc.
>
> Thanks.
>
> --
> ____
> || \\UTGERS,  	 |---------------------------*O*---------------------------
> ||_// the State	 |         Ryan Novosielski - novosirj at rutgers.edu
> || \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
> ||  \\    of NJ	 | Office of Advanced Research Computing - MSB C630, Newark
>       `'
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf