[Beowulf] partitioning L3 by page coloring

[Max R. Dechantsreiter]

Rayson,

> I don't think it can be done if you are not changing the kernel page
> allocator. Physical/virtual page mapping is all done by the kernel -
> in the end, page faults are transparent in the userspace.

This source:

http://users.cis.fiu.edu/~raju/WWW/publications/mascots2011/paper.pdf

suggested to me that there might be simpler approaches:

"Page coloring has been traditionally used to partition CPU caches within the operating system. We use a similar idea to statically restrict the cache available to an application at compile time. Caches in modern systems are designed to be N-way set associative, i.e., a memory line can only be loaded into a set of N available cache lines. The set to which a page is mapped is determined based on a portion of address bits called the index (bits 13 to 24 in POWER6). Partitioning the cache for a specific application can be achieved by controlling this index for all the pages allocated to an application. For instance, allocating pages physically aligned to exactly double the size of a page can effectively halve the number of cache lines available to an application. As shown in Figure 1, this will ensure that bit 21 of physical memory address references issued by the application will always get value 0. Pinning a bit of the index to 0 ensures that the index can no longer take all the 4K values, reducing the available cache sets to 2K only (using the other 11 bits). This directly leads to the application using at most 1MB of the available 2MB cache."

Seems to me this could work with L3 that has a power-of-two
size - not sure about, say, the 12MB L3 of "Westmere".

> Even the traditional page coloring needs help from the kernel. There's
> the "Compiler-directed page coloring for multiprocessors" work done by
> Todd C. Mowry (another U of Toronto prof), but I don't recall seeing
> any pure userspace page coloring techniques - and I would imagine it
> is not entirely possible as the userspace doesn't know the physical
> page address.
>
> http://dl.acm.org/citation.cfm?id=237195

Thanks, I'll have a look.

Rather than make a change to page coloring that could
affect all applications, I would like to apply it to just
one selected application.

Regards,

Max