[Beowulf] Execution time measurements (fwd)

[Christopher Samuel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/03/11 16:26, Mark Hahn wrote:

> if the app doesn't control this (or you with numactl),
> then you should expect performance to lie somewhere
> between the two extremes (fully local vs fully remote).
> the kernel does make some effort at keeping things local
> - and for that matter, avoiding moving a process among
> multiple cores/sockets.

It's worth also mentioning the issue of "NUMA diffusion"
through swapping made by David Singleton from ANU on the
hwloc-devel list:

http://www.open-mpi.org/community/lists/hwloc-devel/2011/02/2012.php

# Unless it has changed very recently, Linux swapin_readahead
# is the main culprit in messing with NUMA locality on that
# platform. Faulting a single page causes 8 or 16 or whatever
# contiguous pages to be read from swap. An arbitrary contiguous
# range of pages in swap may not even come from the same process
# far less the same NUMA node. My understanding is that since
# there is no NUMA info with the swap entry, the only policy
# that can be applied to is that of the faulting vma in the
# faulting process. The faulted page will have the desired NUMA
# placement but possibly not the rest. So swapping mixes
# different process' NUMA policies leading to a "NUMA diffusion
# process".

Keep in mind that the reason that ANU runs systems with swap
is so they can suspend jobs, page the entire thing out and
start a new higher priority job.  Running without swap isn't
really an option for them..

cheers,
Chris
- -- 
    Christopher Samuel - Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
         http://www.vlsci.unimelb.edu.au/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2UYZYACgkQO2KABBYQAh/YxwCdFXz5yAdpKqz3G/Sk+AY73E2E
XmMAn0aws1puNktvnaHQbhycul7pMrRp
=jke/
-----END PGP SIGNATURE-----