[Beowulf] how cluster's storage can be flexible/expandable?

[Vincent Diepeveen]

On Nov 13, 2012, at 9:55 PM, Greg Keller wrote:

>
>
> On Tue, Nov 13, 2012 at 2:40 PM, Vincent Diepeveen <diep at xs4all.nl>  
> wrote:
>
> On Nov 13, 2012, at 9:17 PM, Greg Keller wrote:
>
>
> From: Joe Landman <landman at scalableinformatics.com>  That's not the  
> issue with glusterfs.  It's distributed metadata architecture is a  
> double edged sword.  Very good for distributed data, very very bad  
> for metadata heavy ops.
>
> That and the xfs attributes haven't been slow in years though some  
> folks like bringing up the old behavior pre 2.6.26 as examples of  
> why you shouldn't use it.   Dave Chinner has a great presentation  
> on the topic from 15 months ago or so.  Puts down real numbers.   
> Situation is rather different than implied.
>
>
> We've recently seen XFS kill a pretty important server after an  
> abrupt power off.  It appears someone decided they needed to force  
> it to be "POSIX" compliant by default, and as a result XFS doesn't  
> sync/flush to disk unless told to or some rather long timeout (30  
> seconds can be verrry long ).
>
> You sure this is just XFS problem?
>
> Linux also has this behaviour towards floppy disk over here and  
> that floppy disk doesn't have XFS.
>
> Maybe it's a kernel feature that assumes most people have a cheap  
> WD disk?
>
> In fact to save the new generation harddisks from for example WD  
> that could be interesting feature.
>
> I've read some reports that the cheap versions of them quite  
> quickly go to power savings mode, though not sure how long it takes
> to reach that idle mode. Then it would spin up again when there is  
> disk activity. Yet this flip between power saving and spinning active
> if i see some users report it, that can be done only a limited  
> amount of time, which would normally spoken give the disk because of
> that an average lifetime of no more than a year or so in case your  
> machine is writing regurarly something to disk.
>
> Maybe this feature of XFS helps a tad there? Or should it wait for  
> 30 minutes then to write to disk?
>
> No, I was told this was a conscious decision to follow the POSIX  
> spec exactly.  Maybe it helps not spin up drives and saves $0.07  
> cents per year... but I think that's collateral savings.
>
> To Mr. Becker's Point, we used XFS for large filesystems for years  
> with absolutely no trouble, on 100's of TB of storage.  It was  
> years ahead of EXT4 in making big volumes simple too EXT# types.
>
> Whatever was changed recently (not sure when), I would use XFS, but  
> only once I really understood the tuning/flushing and did some  
> sudden power off and kernel crash testing.  If you're running a  
> version from before the change it will probably work just like you  
> think it should.
>
> Cheers!
> Greg

Heh Greg - didn't ext2 sync every 30 seconds as well?

So if they changed it for XFS as well i guess some old school guy  
hacked a tad around in the code - this is the problem of all  
companies with performance code - too many dudes who aren't thinking.  
One of them can mess up for entire team.