[Beowulf] LXD containers for cluster services and cgroups?

[Lance Wilson]

Hi John,
In regards to your Singularity question we are using cgroups for the
containers. Mostly the containers are used in Slurm jobs which creates the
appropriate cgroups. We are also using the gpu driver passthrough
functionality of Singularity now for our machine learning and cryoem
processing containers which have the cgroups applied to gpus.

Back to your systems containers questions many of our systems have been put
into docker containers as they run on same/similar operating system and
still need root to function correctly. Pretty much every new system thing
we do is scripted and put into a container so that we can recover quickly
in an outage scenario and move around things as part of our larger cloud
(private and public) strategy.

Cheers,

Lance
--
Dr Lance Wilson
Senior HPC Consultant
Ph: 03 99055942 (+61 3 99055942
Mobile: 0437414123 (+61 4 3741 4123)
Multi-modal Australian ScienceS Imaging and Visualisation Environment
(www.massive.org.au)
Monash University

On 15 June 2017 at 20:06, John Hearns <hearnsj at googlemail.com> wrote:

> I'm not sure this post is going to make a lot of sense. But please bear
> with me!
> For applications containers are possible using Singularity or Docker of
> course.
>
> In HPC clusters we tend to have several 'service node' activities, such as
> the cluster management/ head node, perhaps separate provisioning nodes to
> spread the load, batch queue system masters, monitoring setups, job
> submission and dedicated storage nodes.
>
> These can all of course be run on a single cluster head node in a small
> setup (with the exception of the storage nodes).  In a larger setup you can
> run these services in virtual machines.
>
> What I am asking is anyone using technologies such as LXD containers to
> run these services?
> I was inspired by an Openstack talk by James Page at Canonical, where all
> the Opestack services were deployed by Juju charms onto LXD containers.
> So we pack all the services into containers on physical server(s) which
> makes moving them or re-deploying things very flexible.
> https://www.youtube.com/watch?v=5orzBITR3X8
>
> While I'm talking abotu containers, is anyone deploying singularity
> containers in cgroups, and limiting the resources they can use (I'm
> specifically thinking of RDMA here).
>
>
>
> ps. I have a terrible sense of deja vu here... I think I asked the
> Singularity question a month ago.
> I plead insanity m'lord
>
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20170616/f97cc909/attachment.html>