[Beowulf] [gorelsky at stanford.edu: CCL:dual-core Opteron 275performance]

[Alan Louis Scheinine]

  1) Gerry Creager wrote "Hoowa!"
     Since the results seem useful, I would like to add the following.
     On dual-CPU boards with Athlon32 CPUs, the program "bolam" was slow if
     both CPUs on the board were used, it was better to have one MPICH process
     per compute node.  This problem did not appear in another cluster that had
     Opteron dual-CPU boards (single-core), that is, two processes for each node
     did not cause a slowdown.  This is an indication that "bolam" is at a
     threshold for memory access being a bottleneck.  A complication for this
     interpretation is that the Athlon32 nodes use Linux kernel 2.4.21.
  2) Mikhail Kuzminsky asked "do you have "node interleave memory" switched off?
     Reading the BIOS:
     Bank interleaving "Auto", there are two memory modules per CPU so there
        should be bank interleaving.
     Node interleaving "Disable"
  3) In an email Guy Coates asked
     > Did you need to use numa-tools to specify the CPU placement, or did the
     > kernel "do the right thing" by itself?
     The kernel did the right thing by itself.
     I have a question: what are numa-tools?
     On the computer I find
     man -k numa
        numa   (3)  - NUMA policy library
        numactl(8)  - Control NUMA policy for processes or shared memory
     rpm -qa | grep -i numa
        numactl-0.6.4-1.13
     Is numactl the "numa-tools"?  Is there another package to consider installing?
     I see that numactl has many "man" pages.

Reference, previous message:
 >In all cases, 4 MPI processes on a machine with 4 cores (two dual-core CPUs).
 >Meteorology program 1, "bolam"    CPU time, real time (in seconds)
 >      Linux kernel 2.6.9-11.ELsmp     122        128
 >      Linux kernel 2.6.12.2            64         77
 >
 >Meteorology program 2, "non-hydrostatic"
 >      Linux kernel 2.6.9-11.ELsmp     598        544
 >      Linux kernel 2.6.12.2           430        476


-- 

  Centro di Ricerca, Sviluppo e Studi Superiori in Sardegna
  Center for Advanced Studies, Research, and Development in Sardinia

  Postal Address:               |  Physical Address for FedEx, UPS, DHL:
  ---------------               |  -------------------------------------
  Alan Scheinine                |  Alan Scheinine
  c/o CRS4                      |  c/o CRS4
  C.P. n. 25                    |  Loc. Pixina Manna Edificio 1
  09010 Pula (Cagliari), Italy  |  09010 Pula (Cagliari), Italy

  Email: scheinin at crs4.it

  Phone: 070 9250 238  [+39 070 9250 238]
  Fax:   070 9250 216 or 220  [+39 070 9250 216 or +39 070 9250 220]
  Operator at reception: 070 9250 1  [+39 070 9250 1]
  Mobile phone: 347 7990472  [+39 347 7990472]