[Beowulf] Containers in HPC

[Prentice Bisbal]

If you understand the difference between processes and threads, VMs are 
like processes, whereas containers are analogous to threads.

On 5/23/19 8:23 AM, Bill Broadley wrote:
>
> On 5/23/19 3:49 AM, Jonathan Aquilina wrote:> Hi Guys,
>>
>>
>> Can  someone clarify for me are containers another form of virtualized systems?
>> Or are they isolated environments running on bare metal?
> Generally virtual machines run their own kernel.  Typically CPU overhead is
> close to zero, but things like network or disk I/O can be heavily impacted.  VMs
> also typically require carving out a chunk of ram from the host system and
> giving it to the guest.  So the memory overhead is inflexible, and mostly
> static.  There are workarounds (like balloon memory drivers), but generally the
> memory overhead is high.  Virtual machines also boot much like a regular OS, 10s
> of seconds to minutes is common.
>
> Containers do not involve a second kernel, but instead use cgroups (or similar
> on other platforms) to give a container a chunk of system resources.  This makes
> it easy to run a container expecting a different set of libraries, file system
> layout, accounts, namespace, filesystems, etc to run on the same host.  While
> you can limit the ram allocated to a container, it only has to consume what it
> needs.  Cgroups can limit what a container can do, but generally the isolation
> is not as good as with a virtual machine.  Containers can launch in a small
> fraction of a second.  One experiment I did ran fedora, rhel, and ubuntu
> containers and ran "uname -a" or equivalent in all 3.  I was able to launch all
> 3, get the output, and shut them all down in under 1 second.
>
> The I/O and network overhead of containers is minimal, because you are using the
> same kernel.  To the host kernel the difference between a container and a
> process is minimal.
>
> To further confuse things, often people end up running a collection of
> containers in a virtual machine.  Kubernetes (and many other platforms) can use
> this model.  But you can run containers on "bare metal", without using any
> virtual machine, just directly on the underlying OS.
>
> Hopefully that helps.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf