[Beowulf] Re: UPS & power supply instability

[Robert G. Brown]

David Kewley writes:

>> This is a classic symptom of cheap, poorly designed and made power
>> supplies. Or bad room wiring, with undersized neutral lines.
> 
> The PDUs have a front panel that displays lots of diagnostic measurements, 
> and they sound a rather piercing alarm when any measurement goes over its 
> Liebert-defined limit (they are the only alarms I've heard in that room 
> that can reliably be heard over the room noise, from any part of the 
> room :).  The PDUs also have suitably sized breakers and suitably sized 
> conductors on each of the 93 branch circuits.
> 
> The three output phase currents all stay well under their limits, even when 
> they begin to become unstable (at the low-power end of the instability, and 
> well into the instability domain).  Toward the high-power end of the 
> instability domain that we've tested, the current oscillations become large 
> enough, and sit on top of a large enough average current, so the PDUs *do* 
> give overcurrent alarms (plus other alarms due to the wild oscillations).
> 
> Unless something is going on that is not alarmed for, the PDUs and the Liert 
> techs who've been onsite don't indicate any problem with the neutral wiring 
> or the power supplies per se.

I don't understand.  Didn't you say in the original post that under
balanced load there was a substantial neutral line current?

To make things perfectly clear -- with a balanced load and PF 1 and no
harmonic distortion -- the neutral line current should perfectly cancel.
Even with small variations -- the same number of nodes turned "on" on
each phase but maybe not in identical state -- you should cancel to a
few per cent.

If there is a neutral line current AT ALL during balanced operation
there is very likely a problem with the PC power supplies, in my
opinion, the only question is where.  I liked Jim's idea of trying a
balanced, large resistive load on the lines.  A heater, an electric
range.  I also liked the idea of putting a 'scope on the lines because I
mistrust the KAW for things like PF testing, although some of the stuff
reported here suggests that maybe it is ok.  I'd still worry that what
it measures is phase alignment at 60 Hz, not for waveform distortion
(basically additional fourier components at higher frequencies, in
particular 180 Hz).  A scope and measure of neutral line currents could
help.

IF the KAW is indeed insensitive to harmonic distortion, your symptoms
are very, very similar to what we experienced under similar
circumstances with switching power supplies on multiple phases with
shared neutrals and what the Mirus FAQ predicts will happen under those
cases.  Hoofbeats equals horses unless proven otherwise (there ARE
zebras, they're just locally rare, right?:-).

Also, seriously, the phases shouldn't share a neutral and if the cluster
is wired so that they do, this is a design bug all by itself -- this is
just a bad idea for loads that carry harmonics OR are phase shifted
relative to the voltage.  The neutral line builds up a resistive back
voltage proportional to line current times wire resistance -- hence
instead of running at nominal "120V" you might measure a loaded line
voltage of 110V.  

For a resistive load, this has the effect of reducing nominal voltages
by a few volts with no meaningful waveform distortion and any
well-engineered device will tolerate peak voltage variations of maybe
10-15%.  For a third-harmonic distorted load, this back voltage
modulates the neutral line voltage and ADDS to the the 60 Hz phase
signal.  You get beats at 120 Hz, for example.  Except that there are
more harmonics -- the neutral line voltage isn't a pure sinusoid -- and
ANY of those harmonics could couple with accidental resonances anywhere
in your system that the power supply and system designers couldn't have
imagined being present in the system and didn't engineer any defenses
against (not like high frequency surges and noise, for example, that is
pretty well defended even without a surge protector).

Think of it as putting a signal order volts with lots of LOW frequency
noise right across the hot/neutral lines of your system power supplies.
The noise maybe affects the point in the switching cycle where the
system draws current, and you get real fed-back oscillation where the
trigger point rings and resonates.  

I mean, if I grabbed a lab frequency voltage generator and set it to 3V
and 180 Hz and tried to hook it up across your power supply inputs,
you'd hurt me, right?  who would do this and expect things to work
right?  It might well go right through your de facto system high pass
filters designed to keep out "surges" because the impedence of your
power supply capacitors at these low frequencies is not that high.
Maybe it finds something in the natural frequencies of your PC power
supply to resonate with by screwing with trigger points (where phase A
is dropping its current draw right where phase B is trying to raise its
current draw). Maybe you see a 180 Hz signal floated on top of your
supposedly flat line voltages at various places on your mother board.
Maybe you just brown out the supply so your motherboards run a bit under
their nominal voltage.  What GOOD thing could come of it?

    rgb
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20050929/0275cde6/attachment.sig>