[Beowulf] OS for 64 bit AMD

[David Kewley]

Joe Landman wrote on Friday 01 April 2005 22:44:
> Hi David:
>
>    I wasn't challenging you, I just had trouble understanding some of
> what you were saying.

Thanks. :)  I didn't express myself well a couple of times there.

> David Kewley wrote:
> > I am at this moment building a NAS server, and am working to enable and
> > test xfs on RHEL4, for a ~8TB data volume.  I *want* to try xfs on RHEL4,
> > and I hope to use it in production if I can satisfy myself that it's the
> > right choice.
>
> Use FC-3++ and install with the xfs option.  This will allow you to
> format your large area as either ext3 or xfs.

FC gives about 1 year of updates from RH, then another year and a half of 
slowly-released, community-generated updates from the Fedora Legacy project.  
RHEL gives me 5 years of RH updates.  For servers, I'll use RHEL so I can 
plan never to upgrade the OS, just retire the hardware eventually.

After a few attempts at enabling xfs in RHEL4's kernel config, I am still 
figuring out how to stop 'make oldconfig' from taking my CONFIG_XFS_FS=y and 
making it into =n. :/  I'll solve it shortly I'm sure.  Others have reported 
that xfs works fine in RHEL4.

> I have had problems in the recent past (early february) with a drive
> that required some dumps.  This disk had both xfs and ext3 file systems.
>   I was able to get reads of some of the partitions.  I could not
> recover the ext3 file system no matter what I tried (dump and quite a
> few other things).  I was able to recover the xfs file system fairly
> easily.
>
> YMMV.  Mine did.

OK, thanks.

> > I'd really like to see comparisons of TODAY's xfs with TODAY's ext3,
> > using whatever tests or comparisons you find most meaningful.
>
> Last I checked, my systems are running todays versions of everything.  I
> don't necessarily consider steaming bits in CVS/subversion to be todays
> bits.

htrees have been in 2.6, but reservations and online resize were merged only 
in 2.6.10.  Have you tested ext3 on 2.6.10?

> I would suggest that you get the applications list from your users.  As
> many have stated in the past, and many will again state in the future,
> the only meaningful benchmarks are your real codes.  Anything else
> generates waste heat in the universe.

I'll do that, if I can convince my users to wait a while longer before I 
commission their new fileserver...

> > RHEL4 beta versions were available for public download starting 9/27/04;
> > interested parties could have started making detailed comparisons then.
> >
> > Even though I wish xfs was supported in RHEL4, I'm not convinced that
> > RH's choice *not* to support xfs is ill-considered or short-sightedly
> > self-interested or done only out of competitive considerations.
>
> If you read the forums as you pointed out later, you will note that
> there has been a large chorus for quite some time about this.  Redhat's
> response to this chorus has been "wait, ext* will get better".  Yes, it
> has gotten better, but no, this is not what the customers asked for.

In your experience, do customers want certainly performance characteristics 
(which could conceivably be satisfied by ext3), or do they want e.g. xfs 
regardless of actual performance?

>  > I find the
> > argument reasonable that RH chose to stick with ext3 because they believe
> > it meets all their customers' needs,

Let me rephrase that: ...because they believed that they could make ext3 meet 
all their customers' needs in time for RHEL4.

> I am astounded... when the customers in the threads you mention, and in
> the preceding threads made it clear over the last few years that it did
> not indeed meet their needs, and can they please have support for
> something that does ...

Sometimes the customers say they want **xfs**.  RH is ignoring that request.

Sometimes the customers say they want some of the features that xfs provides, 
e.g. large filesystems, high performance, growing filesystems, etc.  RH says, 
"Fine, we'll give you that in ext3."  I do not know whether RH has succeeded 
in meeting customers' feature expectations with ext3 in 2.6.10.  Do you?

> I don't think that is a reasonable argument 
> that what they provide is good enough and meets needs, when clearly
> specific large customers are asking for something different.

If ext3 in RHEL4 fails to meet performance needs of RH customers, then I will 
agree with you.  To my knowledge, the jury isn't in yet, given changes that 
were just merged in 2.6.10.

> > and they don't want to invest in talent to
> > support other enterprise filesystems for major support-purchasing
> > customers.
>
> Again, if small one-man distros can support the system, the argument
> rings hollow.

Either they're stonewalling, or fully supporting (not just supplying, but 
after-sales *supporting*) a new filesystem is a major undertaking that they 
deemed unnecessary.  I don't know which is the case; Arjan says the latter.

> > If you believe he's misguided, argue with what he wrote. :)
>
> Don't need to.  This has been debated in various threads for the last 4+
> years.  Ext3 will continue to improve, and it is probably good enough
> for some section of users.  For others, who need what xfs provides, ext3
> is not sufficient.

What does xfs provide that ext3 in 2.6.10 doesn't?  I ask sincerely.  I 
suppose at this point you could only answer guided by feature lists (which I 
don't know as well as you do).

To answer performance questions, we'd have to test, no?  Or does history 
condemn ext3 to never challenge xfs in those domains where xfs has 
historically beaten ext3? :)

> > How can you say, "Obviously support has nothing to do with it"?  I don't
> > get that.  They have major customers buying expensive support for RHEL. 
> > They offer no paid support for FC.  If they choose, they can ignore bug
> > reports on FC xfs.
>
> https://bugzilla.redhat.com/beta/show_bug.cgi?id=150427
> http://marc.free.net.ph/message/20050310.085034.9d849d03.en.html
>
> Redhat pushes them off to the fs maintainers, as they should.  Bugs
> ignored make for bad practice.

This behavior is consistent with declining to develop in-house xfs expertise, 
no?

> If it were a support issue for RH, it should be a support issue for
> SuSE, for Mandrake, for Debian, for ....
>
> But it isn't.

Leave Debian out of it -- the Debian maintainers don't offer support 
contracts.

So let's see how SuSE and Mandrake respond to bug reports in major components 
like filesystems.  Do they never say, "Sorry, we can't help you with that, 
please go upstream" with an implicit "because we don't have in-house 
expertise"?

Unfortunately, I don't see a quick way to find out an answer to this question.  
Quick look-arounds on the SuSE and Mandrake sites haven't revealed to me 
anythink like bugzilla.redhat.com.  Are there near-equivalents, where we can 
see community bug reports, and read corporate engineers' responses?



Again, my main point is that testing is needed to see whether ext3's 
developers have made major strides in 2.6.10, or have once again failed to 
live up to promises.  Anything else is speculation (which we're doing a 
pretty good job of in this subthread :).

David