[Beowulf] urgent: cost of fire suppression?

Wed Apr 20 08:15:09 PDT 2016

Combustion not supported?
What concentration is that, but it could be pretty low and things would still smolder (e.g. Underground coal seams burning for decades/centuries), but I guess, if the goal is “get the people out” then that’s ok, but as a “preserve the hardware” I don’t know if it would work.

Let’s think about why you might have a fire event in a computer center:

  1.  overheating of some electronic component – maybe capacitors in a power supply, or a bad plug/receptacle.  The thing that gets hot is destroyed, and maybe stuff around it.  Would it spread?  Somehow modern racks of computers may or may not be a good propagation medium.  Is the cable insulation flammable?  Cheap PVC sure is, but you’d probably have a bigger problem from the decomposition products than from the fire.  Plenum rated cables are fire retardant, of course.
  2.  Flammable debris or structure catches fire: either from an accident (dropping the lit cigar into the trash bin full of shredded confidential printouts?) or deliberate (arson, probably more likely than accident).  Is Halon/Mist/low O2 going to help this?  Maybe, but the low O2 would turn it into a smolder, until someone opens a door, and then it might just flashover spectacularly into a burning conflagration
  3.  The building/office next door catches fire and the radiant heat causes your building to catch fire (or the fire simply spreads).  Last year, someone set a largish wooden framed building under construction on fire in Downtown Los Angeles, and the radiant heat broke windows and started fires in adjacent buildings.   Nuclear detonation can cause the same effects from the thermal pulse (just covering all the bases, however unlikely in practice)

In any case, I’m not sure that preservation of the hardware is a realistic goal here.  (recall that the data can be preserved by the simple means of real-time off site backups.. we’re not worrying about saving the tape library).  Or, at least that convention “emergency power off and run the conventional fire sprinklers”.  On more than one occasion, I’ve been in a room where the sprinklers went off, and it is a real deluge: on the other hand, I don’t think any of us were impaired in getting out the door (aside from wet stuff everywhere, it makes a spectacular mess of anything in the room, and it’s a “inches a minute” sort of flow rate.  (something like C in gallons/min = 18 *F *Sqrt(A), where A is in square feet, F is a factor depending on the construction, ranging from 1.5 (wood frame) to 0.6 (fire resistive).

So a 20x50 foot room (1000 square feet) with a factor of 1, would require 500 gallons/minute (the minimum).  that’s 67 cubic feet per minute which is 0.8 inches/minute.   (I’m sure there’s a SI unit version of all of this but building codes in the US tend to use U.S. Customary Units, so that’s what I have handy)
(there’s a bunch of other factors that can be applied for how exposed you are to other buildings, or whether it’s a big open space or lots of connected spaces)
There’s a whole lot of other design stuff (get out your Moody diagram for pipe flow calculation) for things like spacing of the sprinklers, whether you have a dry or wet system, whether there’s pre-charge, etc.

Anyway, in a very few minutes, you’re standing in a lot of water which is running out the doorway as you rapidly make your way out to turn off the sprinklers (in my own experience, there wasn’t any actual fire when the sprinklers went off)

This water flow will thoroughly soak all your hardware, and you might as well consider it a write off.  You could probably wash it and dry it if needed, but computers are pretty cheap these days: I think the labor to rinse all the hardware in deionized water and clean all the gunk off (fire sprinkler water is not “clean water” in general) would cost more than just buying new hardware: you’re going to be on the hook for the labor to remove and reinstall stuff in the racks anyway. And the prospect of climbing around in the subfloor redoing all the cabling amid all the water and residue is pretty ugly.  I’ll bet anyone faced with this would scrap it all to bare walls and build again: a few guys and gals in hazmat suits and the indoor equivalent of a skip-loader to scrape and shovel the remains of your cluster out the doors where it can be safely disposed of as hazardous waste.

Jim

From: Beowulf <beowulf-bounces at beowulf.org<mailto:beowulf-bounces at beowulf.org>> on behalf of John Hearns <hearnsj at googlemail.com<mailto:hearnsj at googlemail.com>>
Date: Tuesday, April 19, 2016 at 11:57 PM
To: "beowulf at beowulf.org<mailto:beowulf at beowulf.org>" <beowulf at beowulf.org<mailto:beowulf at beowulf.org>>
Subject: Re: [Beowulf] urgent: cost of fire suppression?

Tim, the water mist type systems looked interesting - and are claimed to do no damage.

There is also a university HPC data centre in a rival city to yours not far away, where the server room has argon injected to keep the oxygen level below the point where combustion is not supported.
You can work in the server room, but are not allowed to be alone in the building.
I worked in there many years ago.

On 19 April 2016 at 23:55, Lux, Jim (337C) <james.p.lux at jpl.nasa.gov<mailto:james.p.lux at jpl.nasa.gov>> wrote:
And really, really expensive to replace.

For just that Montreal Protocol reason.

Besides, you have good backups and checkpoints, right?  If your cluster
catches fire, you order up a new cluster ³from the cloud² and continue
work.  Doesn¹t Amazon deliver these with big autonomous octocopters
now?<grin>

But realistically,the need for ³non-damaging fire suppression² has gone
away for a lot of data centers, since they have to have good disaster
response plans with very fast response times compared to the 70s and 80s
when ³batch² ruled the day. Imagine if you¹re handling stock transactions,
or even something as mundane as home loans, with fairly tight time limits
and downtime requirements. The regulators aren¹t going to be interested in
your story about how you had all that Halon in your one data center, and
then a flood wiped you out.

If you¹ve got a geographically dispersed hot standby (or even just load
sharing), you can use water to put the fire out enough to save lives, and
let insurance haggle about the equipment replacement.

On 4/19/16, 1:39 PM, "Beowulf on behalf of Greg Lindahl"
<beowulf-bounces at beowulf.org<mailto:beowulf-bounces at beowulf.org> on behalf of lindahl at pbm.com<mailto:lindahl at pbm.com>> wrote:

>On Tue, Apr 19, 2016 at 07:32:38PM +0200, Per Jessen wrote:
>
>> I thought halon gas was the usual choice for datacentres, has that gone
>> out of fashion?
>
>It was quite popular. However, it's not friendly to the ozone
>layer... which means it's phased out due to the Montreal Protocol.
>
>-- greg
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org<mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
>To change your subscription (digest mode or unsubscribe) visit
>http://www.beowulf.org/mailman/listinfo/beowulf

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org<mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20160420/8651e3ef/attachment-0001.html>