[Beowulf] size of swap partition

[Mark Kosmowski]

> Message: 5
> Date: Tue, 10 Jun 2008 00:58:12 -0400 (EDT)
> From: Mark Hahn <hahn at mcmaster.ca>
> Subject: Re: [Beowulf] size of swap partition
> To: Gerry Creager <gerry.creager at tamu.edu>
> Cc: Mikhail Kuzminsky <kus at free.net>, beowulf at beowulf.org
> Message-ID:
>        <Pine.LNX.4.64.0806092012030.29774 at coffee.psychology.mcmaster.ca>
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
>
> > We have the potential to have to swap whole jobs out of memory on a complete
> > node.
>
> that was our intent as well.  among other things, this scheme enables
> running the cluster "split-personality" - mostly shorter/smaller even
> interactive jobs during the day, with big/long jobs running at night.
> unfortunately, you need a smart scheduler to do this, and ours is dumb.
>
> >> beleive, it is 2 or more GB per core; we have 16 GB per dual-socket
> >> quad-core Opteron node). What is typical modern swap size today?
>
> are you willing to use a node which is actually occupying 16 GB of swap?
>
> it is possible to tune how the kernel responds to memory crunches -
> for instance, you can always avoid OOM with the vm.overcommit_memory=2
> sysctl (you'll need to tune vm.overcommit_ratio and the amount of swap
> to get the desired limits.)  in this mode, the kernel tracks how much VM
> it actually needs (worst-case, reflected in Committed_AS in /proc/meminfo)
> and compares that to a commit limit that reflects ram and swap.
>
> if you don't use overcommit_memory=2, you are basically borrowing VM
> space in hopes of not needing it.  that can still be reasonable, considering
> how often processes have a lot of shared VM, and how many processes
> allocate but never touch lots of pages.  but you have to ask yourself:
> would I like a system that was actually _using_ 16 GB of swap?  if you
> have 16x disks, perhaps, but 16G will suck if you only have 1 disk.
> at least for overcommit_memory != 2, I don't see the point of configuring
> a lot of swap, since the only time you'd use it is if you were thrashing.
> sort of a "quality of life" argument.
>
> >> But what are the reccomendations of modern praxis ?
>
> it depends a lot on the size variance of your jobs, as well as
> their real/virtual ratio.  the kernel only enforces RLIMIT_AS
> (vsz in ps),assuming a 2.6 kernel - I forget whether 2.4 did
> RLIMIT_RSS or not.
>
> if you use overcommit_memory=2, your desired max VM size determines
> the amount of swap.  otherwise, go with something modest - memory size
> or so.  but given that the smallest reasonable single disk these days
> is probably about 320GB, it's hard to justify being _too_ tight.

Is anyone using those Gigabyte i-RAM type devices from swap?  Or is
RAM cheaper?  What about using these devices as swap to "add RAM" to
older equipment that is at the maximum mobo supported RAM limit?