[Beowulf] Problem with ext3 filesystem?

[Peter Kjellstrom]

On Thursday 12 February 2009, Joe Landman wrote:
> Dan.Kidger at quadrics.com wrote:
> > And anyway - I thought the maximum size for ext3 was 8TB ?
>
> It is w/o the patches.  With patches it can get to 16TB.  I seem to
> remember that the 16TB required an 8kB page size (e.g. Itanium2 kernel).
>
> > I know that there are patches to bring it up to 16TB, but does
> > anybody trust these and use them for production systems?
>
> I am sure a few people are using this.  We just don't advise/recommend
> this file system at these sizes.
>
> Lustre uses a (significantly patched) ext3 right now to get beyond 8TB
> per OSD.

Actually Lustre doesn't work with >8T OSTs today (trying will eat your data). 
As I understand it this is not likeley to change until some time during 1.8 
when they rebase on top of ext4.

So, in reality it's the other way around. The Lustre ext3 variant (ldiskfs) 
only goes to 8T while the CentOS/RHEL normal one should go to 16T.

/Peter

> Since they appear to be migrating over to zfs, and their 
> patches aren't in the kernel yet (will go back and recheck), this is
> still a problem.
>
> Since we have customers/users with 40+ TB on single boxen we don't
> normally recommend file systems that run close to their limits in daily
> usage scenarios.  That and fsck ...  well ...
>
> Joe



-- 
------------------------------------------------------------
  Peter Kjellström               | E-mail: cap at nsc.liu.se
  National Supercomputer Centre  |
  Sweden                         | http://www.nsc.liu.se
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20090213/6c28927e/attachment.sig>