[Beowulf] Facebook Adapts Open Compute for Colo Space

[Eugen Leitl]

http://www.datacenterknowledge.com/archives/2012/10/25/facebook-adapts-open-compute-for-colo-space/

Facebook Adapts Open Compute for Colo Space

By: Rich Miller

October 25th, 2012

A rack of Facebook servers in a third-party data center in Virginia, adapted
to work with Open Compute designs normally used in Facebook’s company-built
facilities. (Photo: Open Compute Project)

We’ve been closely tracking the progress of the Open Compute Project,
wondering if these uber-efficient open source hardware designs would ever be
available at your local colocation center. Facebook has now shared details of
its first use of Open Compute hardware in its third-party colo space.

The Open Compute Project (OCP) was launched in April 2011 to publish data
center designs developed by Facebook for its Prineville, Oregon data center,
as well as the company’s custom designs for servers, power supplies and UPS
units. The move was seen as a welcome departure from the historic secrecy
surrounding data center design and operations. But Facebook’s energy-saving
customizations provide challenges in multi-tenant facilities.

Facebook has now adapted Open Compute servers in leased data center space in
Ashburn, Virginia, and is sharing the details. The process took about three
months, and focused on hacking the racks.

“Because we were deploying the servers into rooms designed for standard
19-inch racks, we ended up slightly modifying the standard rack to suit our
needs,”wrote Facebook’s Pete Bratach on the Open Compute blog. Facebook is
also operating its servers using standard power settings instead of the
custom configuration used in its Prineville facility. The OCP power supplies
were versatile enough to manage this change, which involved a slight loss of
efficiency.  

Making ‘Open Hardware’ More Accessible

The ability to implement Open Compute designs in leased data center space is
important because it will eventually make Facebook’s energy-saving
innovations available to a broader range of data center users. In the short
term, it’s welcome news for Facebook and its data center service providers,
as it will make it easier for Facebook to adapt its next-generation
infrastructure in its existing leased space.

This could be important in markets like Ashburn or Santa Clara, Calif., two
key connectivity hubs where Facebook leases large amounts of third-party
space and is unlikely to build its own facility because of the cost of land
and power.

Bratach reviewed the alterations required to adapt the Open Compute servers
to the service provider’s environment. They included:

    Riveting two shelves and side panels from an Open Compute “triplet” rack
to a standard 19-inch rack to provide the right-angle tabs that can
accommodate the OCP v2.0 chassis.  New power strips on the server racks,
which run off 208V power instead of the OCP 277V. “We bought off-the-shelf
strips from Server Technology. Some racks used both master and slave power
strips, while others used only the master (the part number is for the master
power strip is CS-24VYM313, the part number for the slave power strip is
CL-24VYM313),” Bratahch wrote.  New power cords connecting the Open Compute
power supply to the new power strips.

Since the facility has its own UPS battery backup power, there was no need
for the Open Compute 48V battery backup system. Since the Facebook power
supply is rated down to 190V, powering servers with 208V was within the
normal operating range. “But at 208V, the current needs to increase to keep
constant power, which makes the power supply a little less efficient (94.4%
efficient) than when it’s running at 277V,” Bratach wrote.

He noted that it may be possible to install an Open Compute “triplet” rack in
an existing colo center, but users will need to work closely with data center
providers prior to installation. A key issue is the sturdiness of raised
floors and ramps, as both the triplet racks (2,600 pounds) and OCP battery
cabinet (2,200 pounds) weigh in excess of a ton. At 95 inches in height, the
triplet racks are also taller than the 84-inch standard rack.

Facebook didn’t indicate which provider’s data center had been used for the
OCP installation. In December 2011, Facebook said it was working with
landlord DuPont Fabros Technology (DFT) to implement its Open Compute designs
in its Ashburn, Virginia. But Facebook’s other Ashburn landlord, Digital
Realty Trust, is also a member of the open Compute effort and adapting its
multi-tenant data center floor plans to give customers the option of
implementing elements of the Open Compute designs.