[Beowulf] Weird blade performs worse as more cpus are used?

[Faraz Hussain]

Quoting Joe Landman <joe.landman at gmail.com>:

> What does numastat report?  /tmp is a ramdisk or tmpfs?  Are the  
> nodes/cpus otherwise idle?  What does lscpu on a good/bad node report?
>

/tmp is tmpfs. The node is completely idle. lscpu is identical for the  
slow and normal ones as shown below. The numastat output is shown  
after that.

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                20
On-line CPU(s) list:   0-19
Thread(s) per core:    1
Core(s) per socket:    10
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 62
Stepping:              4
CPU MHz:               2499.897
BogoMIPS:              4999.25
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              25600K
NUMA node0 CPU(s):     0,2,4,6,8,10,12,14,16,18

Slow blade
==========
                           node0           node1
numa_hit              4791488397      3134270297
numa_miss                      0               0
numa_foreign                   0               0
interleave_hit             20751           20698
local_node            4791480891      3134244680
other_node                  7506           25617

Normal blade
============
                           node0           node1
numa_hit               148398986       104992773
numa_miss                      0               0
numa_foreign                   0               0
interleave_hit             20737           20712
local_node             148396757       104968673
other_node                  2229           24100