How to tell when a job is swapping?

[Jeff Layton]

Good morning,

   For a while now I've been checking if a job is swapping
on our clusters using bWatch. The nodes are dual CPU
boxes and we run two MPI processes per node. I usually
look at the load on the nodes to see if it is above 3.0
(sometimes our code will peak out at about 2.3) and I
look at the free swap space number (bWatch just cats the
/proc/meminfo file).
   I usually assume that if the free swap space falls below
the maximum and load starts climbing that the node is
swapping. However, when I talk to the user, he states that
the code is running fine and the timing numbers are where
they should be. So, I'm obviously interpreting something
incorrectly (unless the job is really swapping but for some
reason performance is unaffected).
   Does someone give me a could way to check if a job
is swapping? Maybe a URL?

TIA,

Jeff