[Beowulf] LXD containers for cluster services and cgroups?

[John Hearns]

Thanks Josh.  Am I familiar with modifying Python code and PBS hook scripts?
Yes - I have had my head under the hood of PBS hooks for a long time.
Hence the pronounced stutter and my predelection to randomly scream out
loud in public places.




On 16 June 2017 at 15:48, Josh Catana <jcatana at gmail.com> wrote:

> I know they have a canned scheduler hook to run docker. If you're familiar
> with python modifying their code to run singularity shouldn't be difficult.
> I rewrote their hook to operate in my environment pretty easily.
>
> On Jun 16, 2017 4:29 AM, "John Hearns" <hearnsj at googlemail.com> wrote:
>
>> Lance, thankyou very much for the reply. I will look at Docker for those
>> 'system' type tasks also.
>>
>> Regarding Singularity does anyone know much about Singularity integration
>> with PBSPro?
>> I guess I could actually ask Altair....
>>
>>
>> On 16 June 2017 at 01:30, Lance Wilson <lance.wilson at monash.edu> wrote:
>>
>>> Hi John,
>>> In regards to your Singularity question we are using cgroups for the
>>> containers. Mostly the containers are used in Slurm jobs which creates the
>>> appropriate cgroups. We are also using the gpu driver passthrough
>>> functionality of Singularity now for our machine learning and cryoem
>>> processing containers which have the cgroups applied to gpus.
>>>
>>> Back to your systems containers questions many of our systems have been
>>> put into docker containers as they run on same/similar operating system and
>>> still need root to function correctly. Pretty much every new system thing
>>> we do is scripted and put into a container so that we can recover quickly
>>> in an outage scenario and move around things as part of our larger cloud
>>> (private and public) strategy.
>>>
>>> Cheers,
>>>
>>> Lance
>>> --
>>> Dr Lance Wilson
>>> Senior HPC Consultant
>>> Ph: 03 99055942 (+61 3 99055942 <+61%203%209905%205942>
>>> Mobile: 0437414123 (+61 4 3741 4123)
>>> Multi-modal Australian ScienceS Imaging and Visualisation Environment
>>> (www.massive.org.au)
>>> Monash University
>>>
>>> On 15 June 2017 at 20:06, John Hearns <hearnsj at googlemail.com> wrote:
>>>
>>>> I'm not sure this post is going to make a lot of sense. But please bear
>>>> with me!
>>>> For applications containers are possible using Singularity or Docker of
>>>> course.
>>>>
>>>> In HPC clusters we tend to have several 'service node' activities, such
>>>> as the cluster management/ head node, perhaps separate provisioning nodes
>>>> to spread the load, batch queue system masters, monitoring setups, job
>>>> submission and dedicated storage nodes.
>>>>
>>>> These can all of course be run on a single cluster head node in a small
>>>> setup (with the exception of the storage nodes).  In a larger setup you can
>>>> run these services in virtual machines.
>>>>
>>>> What I am asking is anyone using technologies such as LXD containers to
>>>> run these services?
>>>> I was inspired by an Openstack talk by James Page at Canonical, where
>>>> all the Opestack services were deployed by Juju charms onto LXD containers.
>>>> So we pack all the services into containers on physical server(s) which
>>>> makes moving them or re-deploying things very flexible.
>>>> https://www.youtube.com/watch?v=5orzBITR3X8
>>>>
>>>> While I'm talking abotu containers, is anyone deploying singularity
>>>> containers in cgroups, and limiting the resources they can use (I'm
>>>> specifically thinking of RDMA here).
>>>>
>>>>
>>>>
>>>> ps. I have a terrible sense of deja vu here... I think I asked the
>>>> Singularity question a month ago.
>>>> I plead insanity m'lord
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
>>>> Computing
>>>> To change your subscription (digest mode or unsubscribe) visit
>>>> http://www.beowulf.org/mailman/listinfo/beowulf
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20170616/0caeea8b/attachment.html>