[Beowulf] OS for 64 bit AMD
landman at scalableinformatics.com
Fri Apr 1 05:49:22 PST 2005
David Kewley wrote:
> Joe Landman wrote on Thursday 31 March 2005 10:04:
>>Actually Redhat now has HPC pricing per node. There are other good
>>reasons to look elsewhere for HPC distributions though, specifically due
>>to their lack of good high performance/scalable per-node file system.
> In your experience/knowledge, what are the issues that make e.g. xfs better
> than ext3 on RHEL4? I ask specifically about RHEL4 because RH has worked
> hard on ext3 in time for RHEL4.
RHEL4 is too new to talk about conclusively. Oddly enough, the same
language was used for RHEL3, that Redhat has worked really hard on
In real world applications (large data set analysis codes, and related)
it is reasonably well known that you get better performance out of ext3
by turning off journaling, effectively making it ext2. Ext3 has a
number of serialization and blocking issues associate with its
journaliing code, not to mention other issues (directories with many
files comes to mind).
The I/O dominated/intensive applications that I have seen or worked with
using xfs vs ext3 have demonstrated significant performance advantages
to xfs. Due to NDA's I cannot talk about specific details though the
performance differences were significant.
> My understanding is that RH choose to support ext3 but not xfs because: 1)
> they have in-house expertise for ext3 but not for xfs, and 2) they believe
> that xfs has no real advantages over ext3.
They have many customers that disagree with 2, and 1 is a tautology, as
they have invested heavily in ext3, and thus they have people to work on
Some inherent disadvantages of ext3 show up when you start looking at
large file systems and large files. Xfs has much higher limits. If you
want to build a 30TB file system across a huge disk array attached to a
sizeable SMP machine, can you do it with ext3? (no as of RHEL3). If
you want to work with a 2.5 TB file (part of a recent benchmark we ran),
can you do it with ext3? (no as of RHEL3). Xfs doesn't have a problem
with either of these.
> If customers show RH that there are real-life needs for xfs that are not
> satisfied by ext3, then RH may well be willing to invest in in-house xfs
Unlikely. Customers have been showing a clear need for this for a while
(Sloan sky survey, and many others with huge and high speed data
requirements). Redhat prefers to use the excuse that it is a large and
complex package. Hmmm. So are Xorg, Openoffice, ....
I do not expect Redhat to do this. SuSE has, as have most of the rest
of the major distributions (including the 1 man distribution shops), so
the excuses that one hears are ... well ... probably not the real
reasons. Redhat does not want to promote a competitor to technology it
supports. That seems to be a simpler explanation, and I believe is
better supported by observing their actions.
Note: FC-x has xfs enabled, one only needs to use "linux xfs" on the
boot line during installation.
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax : +1 734 786 8452
cell : +1 734 612 4615
More information about the Beowulf