[Beowulf] Servers Too Hot? Intel Recommends a Luxurious Oil Bath

[Joe Landman]

On 09/05/2012 09:25 AM, Prentice Bisbal wrote:
> On 09/05/2012 12:38 AM, Joe Landman wrote:
>> On 09/05/2012 12:28 AM, Lux, Jim (337C) wrote:
>>
>>> But as long as we're talking quarrys and such, what about the scheme of
>>> building a big pit to fill full of ice during the winter, and melting it
>>> during the summer. (assuming you are in a
>>> less-than-wonderful-un-California-like climate where this would work.)
>> I bet a good business case could be made for something like this.  The
>> west coast of Michigan is rich in having good wind availability and an
>> excellent (and visible from space) reservoir of very cold water that
>> could be used in heat exchangers to cool data centers.  Use the cool air
>> during the cooler months, and the cold water during the warmer months.
>>
>> Given the proximity to Chicago, I'd think it would make for a good
>> business to house LOTS of servers in a naturally well air conditioned
>> environment.    I can't imagine that the AC bill in Secaucus and
>> elsewhere in New Jersey is very pleasant.  Could ring the Great Lakes
>> with these.
>>
>> A shame I don't have the capital to do this.  Bet we could sell out the
>> space.  Without using oil, hydrogen/helium, or LN2.
> I don't think that would be a wise investment, since Iceland is poised
> to become the colocation capital, due to it's cold temperatures and
> cheap, environmentally friendly geothermal energy. They're also looking
> to increase the network bandwidth between Iceland and Europe and North
> America. I brought this up on this list a couple of months ago:
>
> http://www.beowulf.org/pipermail/beowulf/2012-April/029613.html
>

Yeah ... but geography/location and other things conspire against this 
becoming anything more than a niche player.

In HFT latency matters.  Really ... seriously ... matters.  36 
milliseconds to get to the DC?  Won't fly.  The folks we work with are 
looking to shave microseconds off everything.  Anything and everything 
is on the table for this.

Extending the distance to your DC by a few thousand miles to take 
advantage of cheaper power while losing out on the fastest trading?  Not 
gonna happen.  The article even notes this:

"That kind of connectivity is helpful, but not perfect. Laursen notes 
that the minimum ping time from Iceland to New York is 36 milliseconds. 
This means that low-latency applications, like high speed trading would 
not be practical.  But for straight-up HPC and generic enterprise 
computing, it’s usually not an issue."

This said, the closeness to various giga-POPs here, the low cost of 
adding fibre, the low cost of cooling, the relative geostability of 
Michigan (and the Great Lakes in general), the political and economic 
stability of the region ... When you consider all these things, it makes 
a great deal of sense to do this relative to other choices.

Yahoo and others realize this: 
http://www.datacenterknowledge.com/archives/2011/04/06/yahoo-gets-power-for-buffalo-expansion/ 
(Buffalo NY is a Great Lakes regional city, and a nice place in its own 
right).

Cheaper power is simply one aspect of the cost of doing business.  But 
its not the only one.  The cost to cool data centers is far more 
important, and why Google, Yahoo, et al. strive for PUE ratings close to 
1 (c.f. http://en.wikipedia.org/wiki/Power_usage_effectiveness).  Its 
also why ARM is interesting, as are lower power Intel and AMD offerings. 
  Same power, more clock cycles.  Same power, very little cooling.

Harder than getting low cost power is getting LOTS of power into a data 
center.  If you are at 20MW, and you need another 20MW in for expansion, 
this could wind up costing you far more in construction and 
infrastructure costs than it would in terms of usage costs.


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615