[Beowulf] Good IB network performance when using 7 cores, poor performance on all 8?

[Waleed Harbi]

I believe rh6.5 will give great results. We faced many issues with rh6.1
and 6.2, Redhat advice us to move to the new release. Sometime you need
check the scheduler configuration too. The final thing I suggest, MPI code
optimization.

On Thursday, April 24, 2014, Brian Dobbins <bdobbins at gmail.com> wrote:

>
> Hi John,
>
> I am with Joe regarding looking at the interrupts.
>>
>
>   That was one of my initial thoughts, too, but I haven't really dug into
> their states yet.  I'm going to start keeping an eye on them now.
>
>
>> However, could this be a difference with the power management with the
>> Redhat kernel?
>> ie. when running on 8 cores you are tripping over some thermal threshold
>> and causing a throttle back to a lower C-state?
>>
>
>   This is something I hadn't thought of but which makes a lot of sense,
> and in my reply to Joe, you'll see we've got a power-saving state selected
> for the PCIe ASPM.  I'm going to see if our guys can turn that off and
> whether that helps, and I'll poke around some of the other tunable power
> options soon, too.  I don't have root access and sit more on the
> application side so I'm a bit rusty on all the buttons and dials, but I'll
> investigate a bit and report back what I find.
>
>
>> Can you give the kernel versions for both setups?
>>
>
>   The C5.5 kernel is 2.6.18-194
>   The R6.2 kernel is 2.6.32-220
>
>   There's a few more kernels in the mix now, but I'd like to check out a
> few other things before I add to the confusion.  ;-)
>
>   Thanks again,
>   - Brian
>
>
>

-- 
--------------------------
Cheers!
Dodi

Linkedin: http://goo.gl/3eJ9UI
In business do not look for credits, look for innovation!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20140424/1be3db72/attachment.html>