[Beowulf] [zfs-discuss] ZFS NAS build: hardware recommendations.

[Eugen Leitl]

----- Forwarded message from Liam Slusser <lslusser at gmail.com> -----

Date: Thu, 30 May 2013 13:02:35 -0700
From: Liam Slusser <lslusser at gmail.com>
To: zfs-discuss <zfs-discuss at zfsonlinux.org>
Subject: Re: [zfs-discuss] ZFS NAS build: hardware recommendations.
Reply-To: zfs-discuss at zfsonlinux.org

I've purchased a lot of Seagate hard drives over the last few years for our
hadoop and gluster clusters and have been writing down what fails and when.
 These numbers probably aren't exact but they're pretty close - I'm sure
i've missed a few here and there over the years.

Here are those numbers.

Purchased 680 Seagate ST31500341AS 1.5tb SATA 7200rpm between 2008-2011.
 Of those 680 drives, we've had 66 fail in-warranty and another 60 fail out
of warranty.  1 was DOA.  We only had one drive fail twice which was
in-warranty, i'm not sure how many of the out of warranty drives were RMA
units.  Over-all thats about a 18% failure rate.

Purchased 650 Seagate ST2000DM001 2tb SATA 7200rpm from 2011 on.  Of those
650, we've had 27 fail in warranty and none are out of warranty yet.  1 was
DOA.  Over-all about a 10% failure rate so far.

Purchased 88 Seagate ST2000NM0001 2tb SAS 7200rpm Constellation ES from
2011 on.  Of those 88 we've had 1 failure.

Purchased 104 Seagate ST3300657SS 300g SAS 15k Cheetah from 2011 on.  Zero
failures.

Purchased 54 Seagate ST3600057SS 600g SAS 15k Cheetah from 2011 on.  Zero
failures.

Purchased 82 Western Digital WD1501FASS BLACK 1.5tb SATA 7200rpm.  3
failures.

Purchased 112 Intel x25e enterprise SATA 32g SSDs between 2009-2011.  4
have failed.

Purchased 32 Micron p300 50g and 100g SATA2 SSDs between 2011-2012. Zero
failures.

Purchased 8 Intel 520 240g SSDs between 2011-2012.  One failure.

All the above drives are currently online and working (except the dead ones
of course).  They're all heavily used in a 24/7 environment.  They are all
in a climate controlled datacenter in basically identical 4u whitebox
servers.   You can see that the desktop "AS" models Seagate's and low end
2tb SATA drives are much more prone to failure.  The high end SAS are much
more reliable.

Our Western digital BLACK drives have been extremely reliable, I just wish
their RMA process was easier to deal with or I'd buy more.

We have a lot of SSDs, all higher end models with the exception of the
Intel 520s which is a "desktop" model, and have had great luck with all of
them.  Very little failures and outstanding performance.  They're used
almost exclusively in our database servers (Oracle and Voldemort).  Another
note about the desktop Intel 520s, they're extremely fast and great value
for the money - I'm putting in an order this month for 10 more of the 480g
models.  The 520s are half the price of the older Intel x25e and Micron
P300 enterprise models and 4-7 times bigger.

So the take-away of this is buy higher end SAS if you care, cheap SATA are
cheap but expect a much higher failure rate.

thanks!
liam



On Wed, May 29, 2013 at 12:18 PM, Gordan Bobic <gordan.bobic at gmail.com>wrote:

> On 05/29/2013 08:10 PM, Turbo Fredriksson wrote:
>
>> On May 29, 2013, at 8:57 PM, Liam Slusser wrote:
>>
>>  Either way Seagate's RMA process is top notch and replaces them.
>>>
>>
>> Maybe because they always fail, so the're used to it :). Seagate
>> consumer disks don't have a very good reputation.... ESPECIALLY
>> not the 'green' disks!
>>
>
> Depends on the model.
> ST31000340AS was _terrible_. I still have 6 months warranty left on mine,
> and in the 4.5 years I've had them they've each been replaced twice. I only
> have one left - they've of late been sending me ST31000524AS as
> replacements which have been completely reliable thus far. Reliability
> roughly seems to follow the platter count.
>
> ST31000340AS - terrible
> ST31000333AS - OK-ish
> ST31000528AS - perfect so far
> ST31000524AS - perfect so far
>
>
>  That said, I've been using Seagate (not green) exclusively at home for
>> many, many years, and as long as they survive the first week (25% of
>> the disks I buy are dead on arrival, 25-35% die within the first few days
>> and a few of them die within the first two weeks), they are rock solid!
>>
>
> Failure on my 340AS/333AS models has been mostly random, although a few
> have failed within a week (massive media failure, ran out of spare sectors
> for remapping).
>
>
>  Not fast, but they are plenty fast enough for my use - streaming
>> multimedia
>> to my Popcorn Hour (which suck huge elegant d***s!!).
>>
>
> Performance is dictated by rpm. Everything else is secondary.
>
> Gordan
>

----- End forwarded message -----
-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://ativel.com http://postbiota.org
AC894EC5: 38A5 5F46 A4FF 59B8 336B  47EE F46E 3489 AC89 4EC5