[Beowulf] disabling swap on cluster nodes?

[Mark Hahn]

> enable swap on the same nodes. While swapping (both in and out) the

if there is ever a significant period of time when you're both
swapping out and in, you're probably thrashing.  I'm not advocating
thrashing.  however, it's hard to think of rational arguments against
two particular ways to use swap:

- to permit the kernel to offload cold pages.  this is what happens 
in a normal linux system when you see a occasional bits of SO,
and an accumulation of only a few percent (of ram) in swap.
this implies no or trivial SI traffic, and simply provides more
ram to productive uses.

- to permit suspension/preemption of jobs.  this quite a valuable 
scheduler trick, and works surprisingly well.  the suspended job 
gets swapped out (SO traffic but no SI), then back in (SI with 
no SO traffic).  this is incremental, which is not ideally efficient,
but is pretty nice if the preemptor is not huge.

> latest generation CPU gets to perform probably about the same FLOPS as
> an 8086 with software emulated FP - is this really HPC? It's already

again, I'm not talking about thrashing, wherein the CPU is idled waiting
for pages during a storm of both SI/SO traffic.  non-thrash swap usage
does not imply a waste of CPU.

sometimes I think the superstition against swap reflects an anachronistic
belief that disk IO is inefficient.  (ie, perhaps exposure to PIO controllers.)

> bad enough that the RAM is mostly NUMA these days and the various

what does NUMA have to do with it?  I don't think swap is going to have
some sort of pessimizing-for-numa effect - in fact, I don't see why 
it wouldn't help (if a non-numa-optimal page gets swapped out, the 
thread that faults it back in is presumably more numa-optimal...)

> is now free for others) and the user satisfaction (no swap=faster
> finishing) simultaneously :)

that is a false equivalency.

> For many years now I do not configure swap on my workstations or
> laptops either (using mostly Fedora), as they have enough RAM for a
> full-blown graphical desktop environment and all assorted
> applications. And when I do start to see problems,

how strange that you attribute the problem to the availability of swap.

>> we also run with overcommit=2.
>
> Overcommitting is unfortunately still needed for the older Fortran
> applications which statically allocate large arrays. Yes, some of
> those pesky things are still around...

overcommit=2 and swap go together very well (obviously).