[Beowulf] NUMA zone weirdness

[Greg Lindahl]

Wow, that's pretty obscure!

I'd recommend reporting it to Intel so that they can add it to the
descendants of ipath_checkout / ipath_debug. It's exactly the kind of
hidden gotcha that leads to unhappy systems!

-- greg

On Fri, Dec 16, 2016 at 03:52:34PM +0000, John Hearns wrote:
> Problem solved.
> I have changed the QPI Snoop Mode on these servers from
> ClusterOnDIe Enabled to Disabled and they display what I take to be correct
> behaviour - ie
> 
> [root at comp006 ~]# numactl --hardware
> available: 2 nodes (0-1)
> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11
> node 0 size: 32673 MB
> node 0 free: 31541 MB
> node 1 cpus: 12 13 14 15 16 17 18 19 20 21 22 23
> node 1 size: 32768 MB
> node 1 free: 31860 MB
> node distances:
> node   0   1
>   0:  10  21
>   1:  21  10