[Beowulf] how cluster's storage can be flexible/expandable?

[Greg Keller]

On Tue, Nov 13, 2012 at 2:40 PM, Vincent Diepeveen <diep at xs4all.nl> wrote:

>
> On Nov 13, 2012, at 9:17 PM, Greg Keller wrote:
>
>
>> From: Joe Landman <landman at scalableinformatics.**com<landman at scalableinformatics.com>>
>>  That's not the issue with glusterfs.  It's distributed metadata
>> architecture is a double edged sword.  Very good for distributed data, very
>> very bad for metadata heavy ops.
>>
>> That and the xfs attributes haven't been slow in years though some folks
>> like bringing up the old behavior pre 2.6.26 as examples of why you
>> shouldn't use it.   Dave Chinner has a great presentation on the topic from
>> 15 months ago or so.  Puts down real numbers.  Situation is rather
>> different than implied.
>>
>>
>> We've recently seen XFS kill a pretty important server after an abrupt
>> power off.  It appears someone decided they needed to force it to be
>> "POSIX" compliant by default, and as a result XFS doesn't sync/flush to
>> disk unless told to or some rather long timeout (30 seconds can be verrry
>> long ).
>>
>
> You sure this is just XFS problem?
>
> Linux also has this behaviour towards floppy disk over here and that
> floppy disk doesn't have XFS.
>
> Maybe it's a kernel feature that assumes most people have a cheap WD disk?
>
> In fact to save the new generation harddisks from for example WD that
> could be interesting feature.
>
> I've read some reports that the cheap versions of them quite quickly go to
> power savings mode, though not sure how long it takes
> to reach that idle mode. Then it would spin up again when there is disk
> activity. Yet this flip between power saving and spinning active
> if i see some users report it, that can be done only a limited amount of
> time, which would normally spoken give the disk because of
> that an average lifetime of no more than a year or so in case your machine
> is writing regurarly something to disk.
>
> Maybe this feature of XFS helps a tad there? Or should it wait for 30
> minutes then to write to disk?
>

No, I was told this was a conscious decision to follow the POSIX spec
exactly.  Maybe it helps not spin up drives and saves $0.07 cents per
year... but I think that's collateral savings.

To Mr. Becker's Point, we used XFS for large filesystems for years with
absolutely no trouble, on 100's of TB of storage.  It was years ahead of
EXT4 in making big volumes simple too EXT# types.

Whatever was changed recently (not sure when), I would use XFS, but only
once I really understood the tuning/flushing and did some sudden power off
and kernel crash testing.  If you're running a version from before the
change it will probably work just like you think it should.

Cheers!
Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20121113/14b059ed/attachment.html>