[Beowulf] Big storage
jakob at unthought.net
Thu Aug 30 03:23:24 PDT 2007
On Thu, Aug 30, 2007 at 09:53:04AM +0200, Leif Nixon wrote:
> Jakob Oestergaard <jakob at unthought.net> writes:
> > I'm looking at getting some big storage. Of all the parameters, getting as low
> > dollars/(month*GB) is by far the most important. The price of acquiring and
> > maintaining the storage solution is the number one concern.
> Even at the price of reliability.
Yes, to some extent.
> They are nice, I understand. And seem like a good approach. The
> vendors will likely want to sell you SAN-ified systems with cluster
> file systems and lots of Windows-only management software. It can take
> some convincing to get them to understand that that is actually not
> what you are looking for.
> > Any better ideas? Is there a way to get this more dense without
> > paying an arm and a leg? Has anyone tried something like this with
> > HSM?
> That is probably not worth the bother.
I find it interesting (and surprising) how little people like tape :)
> But something that you have to be prepared for when going to that
> storage volume is that you *will* suffer data corruption at some
> point, and you need to plan for it. See for example
> It's quite possible (though unlikely) for a hard disk to suddenly
> return corrupted data without signalling a read error, and this is a
> possibility that raid controllers typically just ignores. And then you
> have the usual crop of software, firmware and hardware errors that can
> trash your data more or less silently.
It seems ZFS end-to-end checksums is not such a bad idea after all :)
> At a minimum, make sure you keep checksums of all files so you can
> verify their integrity.
Thank you for the feedback!
More information about the Beowulf