[Beowulf] Single threaded memory bandwidth compared between Nehalem-EP and Westmere-EX

[Christopher Samuel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hiya Joe,

On 19/10/13 23:10, Joseph Landman wrote:

> Did you try taskset to pin the CPU and the memory on the same numa 
> node?

I wasn't, but using numactl to do that instead (which has more
powerful options from what I can see) shows that what I'm seeing is
the best case. :-(

if I pin the memory on the Max5 expansion unit I get slightly worse
performance and if I pin it to the RAM hanging off the other socket
it's worse again.

That correlates with what the kernel reports for distances between
NUMA nodes too (cheaper to hit RAM on the Max5 expansion than on the
other socket).

BTW numactl can show you the default binding, which is to the current
node:

[samuel at turpin Stream]$ numactl --show
policy: default
preferred node: current
physcpubind: 1
cpubind: 0
nodebind: 0
membind: 0 1 2

> What does numastat report?

It confirms it's writing only to the home NUMA node.

[root at turpin ~]# numastat stream_c.exe

Per-node process memory usage (in MBs) for PID 19275 (stream_c.exe)
                           Node 0          Node 1          Node 2
                  --------------- --------------- ---------------
Huge                         0.00            0.00            0.00
Heap                         0.00            0.00            0.00
Stack                        0.02            0.00            0.00
Private                   2062.07            0.00            0.00
- ----------------  --------------- --------------- ---------------
Total                     2062.09            0.00            0.00

                            Total
                  ---------------
Huge                         0.00
Heap                         0.00
Stack                        0.02
Private                   2062.07
- ----------------  ---------------
Total                     2062.09


cheers!
Chris
- -- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlJlyNEACgkQO2KABBYQAh9lvwCfaNuwixvlFzP8z4Ejuxlzxjb2
ExUAnjehW/Cc+ejDGmAxAosqxxvhtlns
=GhfV
-----END PGP SIGNATURE-----