[Beowulf] GPFS on Linux (x86)

[Mark Hahn]

> work ok.  They describe it as the SATA disks go 'out to lunch' occasionally 
> and a disk will have a large pause before continuing. Those comments come 
> from a vendor (Be wary).

this is true, sorta.  it wasn't really a big surprise - most of the 
initial generation of SATA disks were designed for non-24x7 workloads,
and if you actually kept them busy for a couple hundred hours, they'd
take a vacation, like it or not.  it didn't help that Linux's MD raid
was intentionally naive about handling this kind of error.

I'm pretty sure I remember some older SCSI disks having that problem too.
but there are obvious fixes, and vendors know about this and treat 
it as a marketing feature, so everyone's happy ;)

(personally, I don't often have SATA disks in raid which are actually
busy all the time.  but we're an academic HPC site, so don't have 
anything like 24x7 DB activity, or CERN-like streams of data...)

> But if my customer wants 15K disks, I am going to get them for him/her.
> I am not going put the effort in convincing them to go with SATA.  You
> pick your battles even when serving them.

I'm thankful that in most ways, I'm a customer, not on the other side ;)

but it's worthwhile to note that SCSI disks are still dramatically 
more expensive than SATA.  yes, there are reasons, but even so,
price/performance is a pretty strong argument...

> Large SATA array vs. Large SCSI array, same vendor.  It appears that the 
> SCSI disks were a bit more reliable.  I wish I had more numbers to back it 
> up, but that is what it seemed like.  I don't buy any of the vendor
> numbers.

well, for what it's worth, my organization bought something like 6k
SATA disks over the past year, and have seen pretty sparse failures.
(unfortunately, our book-keeping is probably not good enough...)