[SPAM] [Beowulf] powering up 18 motherboards

[Robert G. Brown]

On Thu, 17 Feb 2005, Jim Lux wrote:

> 
> ----- Original Message -----
> From: "Alpay Kasal" <eno at dorsai.org>
> To: "'Bari Ari'" <bari at onelabs.com>; "'Jim Lux'" <James.P.Lux at jpl.nasa.gov>
> Cc: "'Dean Johnson'" <dtj at uberh4x0r.org>; <beowulf at beowulf.org>
> Sent: Thursday, February 17, 2005 9:26 PM
> Subject: RE: [SPAM] [Beowulf] powering up 18 motherboards
> 
> 
> > I think you hit the nail on the head Bari, I'm in Brooklyn, New York. So I
> > suppose it should be 15amp circuits but every circuit breaker in the box
> is
> > clearly a 10. This is an old house, seems like any renovations over the
> > years have been only for aesthetics. The wiring in the walls is probably

Circuit breaker?  What's wrong with good old fuses?  Good enough for my
granddaddy.... actually, CB's suggest that the wiring has been messed
with sometime in the last twenty or thirty years.  That's a good thing!
In my last house, the fuses DID feed directly into 14 Ga wire pairs that
went through the house six inches apart in porcelain insulators and
through porcelain tubes -- until I got to it and replaced it all, a
circuit at a time, with 12 Gauge three wire PVC.

The fabric coating those old wires DOES tend to disintegrate after 70
years or so... if your house has it it is probably a midlevel electrical
fire trap.

> > disintegrating - that would explain why the new looking circuit breakers
> are
> > rated for 10 amps.
> 
> Ahhh yes.. New York, where some of the (mostly undocumented) distribution
> wiring dates from Edison himself, and dogs are electrocuted when urinating
> on the street from stray currents in connection boxes.  The wiring is
> probably knob and tube.
> 
> >
> > I think I can get use of 3 circuits which gives me some room to play with
> > all the nodes and hopefully the assortment of switches and power supply. I
> > have to figure out what the draw will be on the rest of the equip. Now
> where
> > the hell am I going to plug in this air conditioner????
> >
> > Any advice on how to gang up 3 10amp circuits into a single 30amp? Sounds
> > like a job for an electrician? Thanks for the help guys.
> 
> Why gang em up?  Just run three extension cords or plug strips, one into

In case Jim didn't make it clear, DO NOT GANG THEM UP.  In principle one
can do this, IF all three of the circuits have the same phase.  If they
don't have the same phase (and aren't correctly connected in phase) you
will observe a brief, interesting flash while the circuit breaker does
bad things when you power up after trying it and see "midlevel
electrical fire trap" above.

However, the "right" way to gang them up is to go to the box and run
brand new, clean, NEC-compliant wire from the box to your cluster
location.  The only thing you'll have to worry about is removing as much
total amperage from the box as you add (or at least, staying withing the
distribution box's total capacity).  Here knowing what your house's
total service capacity is, and what the total capacity is of the main
distribution panel is (hopefully they are the same, but one never knows)
is useful.  Maybe the box already has spare capacity and you just don't
know it.

The rule with electrical wiring is that if you don't know EXACTLY what
you're doing, hire a professional electrician.  That is, if you have to
ask how to gang up circuits (thereby demonstrating an ignorance about 2
and 3 phase delivery, hot, neutral and cold/ground wires, ground loops,
etc) you have an unpleasantly high probabiliby of either killing
yourself or burning down your house (possibly months or years after your
renovation), and are probably breaking the law besides when you do it.

Jim's suggestions below are excellent for living with what you have.
Also consider stripping down the machines.  A cluster node these days
can be built out of a motherboard loaded with CPU and memory and with an
onboard, PXE-capable NIC (or at most a PCI PXE NIC).  No floppy, CD,
hard drive, video card, or other peripheral stuff.  A few weeks ago,
there was a lovely discussion on clusters made by mounting bare
motherboards on shelving and powering them off of shared large power
supplies strung together with simple extensions, three motherboards per
450 W supply.  This saves on both heat and power, as EACH peripheral
draws a base current when turned on, including each power supply.  There
are pictures out there if you look.

If you do this, you'll probably drop your load by as much as 30 watts
per node, and this should be enough to push you safely within the
capacity of your circuits at six nodes per circuit with room to spare.

   rgb

> each circuit. 6 machines on each circuit.  If the startup surge is too much,
> stagger the spin up of the disk drives (maybe some sort of BIOS power
> management option?).  Seriously think about cobbling together some
> collection of relays.   The W.W.Grainger catalog is your friend, even if you
> don't buy your stuff from them, you'll at least have a good description to
> go googling for surplus places.
> 
> If that gets too dicey.. Honda and Yamaha make very nice, quiet inverter
> generators.  2kW for about $700.
> 
> Good luck...
> Jim.
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> 

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu