[Beowulf] big read triggers migration and slow memory IO?
samuel at unimelb.edu.au
Thu Jul 9 23:00:46 PDT 2015
On 10/07/15 07:27, mathog wrote:
> Well, that seems to be it, but not quite with the same symptoms you
> observed. khugepaged never showed up, and "perf top" never revealed
> _spin_lock_irqsave. Instead this is what "perf top" shows in my tests:
So this sounds look like the same symptoms as reported here too:
There they note that:
# Looking at the functions listed in the Flame Graph (below)
# it looked like the issue was related to virtual memory and
# the Linux source shows many of these functions are in
which is indeed where compation_alloc() lives.
The kernel describes that function as:
* This is a migrate-callback that "allocates" freepages by taking pages
* from the isolated freelists in the block we are migrating to.
* Isolate free pages if necessary, and if we are not aborting due to
Introduced by this commit:
Author: Mel Gorman <mel at csn.ul.ie>
Date: Mon May 24 14:32:27 2010 -0700
mm: compaction: memory compaction core
This patch is the core of a mechanism which compacts memory in a zone by
relocating movable pages towards the end of the zone.
A single compaction run involves a migration scanner and a free scanner.
Both scanners operate on pageblock-sized areas in the zone. The migration
scanner starts at the bottom of the zone and searches for all movable
pages within each area, isolating them onto a private list called
migratelist. The free scanner starts at the top of the zone and searches
for suitable areas and consumes the free pages within making them
available for the migration scanner. The pages isolated for migration are
then migrated to the newly isolated free pages.
[aarcange at redhat.com: Fix unsafe optimisation]
[mel at csn.ul.ie: do not schedule work on other CPUs for compaction]
Signed-off-by: Mel Gorman <mel at csn.ul.ie>
Acked-by: Rik van Riel <riel at redhat.com>
Reviewed-by: Minchan Kim <minchan.kim at gmail.com>
Cc: KOSAKI Motohiro <kosaki.motohiro at jp.fujitsu.com>
Cc: Christoph Lameter <cl at linux-foundation.org>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu at jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm at linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
All the best,
Christopher Samuel Senior Systems Administrator
VLSCI - Victorian Life Sciences Computation Initiative
Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
More information about the Beowulf