[Beowulf] OpenMP on AMD dual core processors

Bill Broadley bill at cse.ucdavis.edu
Sat Nov 22 12:45:09 PST 2008

Greg Lindahl wrote:
> On Fri, Nov 21, 2008 at 09:38:29AM -0600, Nathan Moore wrote:
>> I had a similar problem with gfortran, and it only appeared with large array
>> sizes (bigger than 4000x4000 as I recall).  "ulimit" was no help, I assume
>> there's a memory constraint built in somewhere.
> With OpenMP the compiler has to set up multiple stacks, and some are
> more clever than others. If you were using the PathScale compiler, for
> example, and overran one of the thread stacks, it would print out an
> error message saying what happened and how to raise that limit.

I see warnings from Pathscale-3.2 when I run 2 8k x 8k arrays of doubles using
1 or 2 threads (but not 4 or 8 threads):
** OpenMP warning: requested pthread stack too large, using 4294967296 bytes

But it still seems to work.

On my machine (with 8GB ram) I can run 2 8k x 8k arrays of doubles without
problem.  For runs with 8kx8k arrays, convergence_v = 1, 24 iterations and
gcc-4.3.2 -O3 -fopenmp:
real    3m39.818s
real    2m21.298s
real    2m19.850s
real    1m39.412s

Pathscale-3.2 -O3 -mp:
real    3m10.803s
real    2m24.492s
real    1m43.183s
real    1m20.669s

I believe that arrays in C/Fortran have to be contiguous by default and that
depending on kernel (32 vs 64bit), PAE, and BIOS settings sometimes all
physical memory isn't contiguous.

More information about the Beowulf mailing list