[Beowulf] Network Filesystems performance
landman at scalableinformatics.com
Sun Nov 18 17:58:30 PST 2007
Ashley Pittman wrote:
> On Sat, 2007-11-10 at 18:33 +1100, Chris Samuel wrote:
>> On Fri, 24 Aug 2007, Michael Will wrote:
>>> The first advise is to stay away from redhat for file servers since
>>> they have some bursty I/O bugs and don't support XFS.
>> We've run through Fedora (OK, but release cycle too quick), Ubuntu
>> (Dapper has longer support, but a bit flakey) and now run Debian Etch
>> (very happy).
We have used SuSE (all flavors), RHEL/Centos (many flavors), Ubuntu, and
a few others. We have generally found that the RHEL requires the most
modifications to get to a reasonable base level (the Rocks folks are
dealing with the issues of new hardware/old kernel right now, and it
isn't pretty). SuSE requires some tweaking (and sometimes it is
annoying to deal with all the added "extra bonus [mis]features" like
zenworks/zmd). Ubuntu has been by far the easiest to make "just work"
and work well at that. I know it is Debian under the hood. I know some
folks like Fedora as well.
Most distro discussions devolve into pitched emotional battles at some
point, full of an overabundance of heat, and an underabundance of light.
> This is an interesting proposition, you seem to imply it's easier to
> change distribution that it is to customise the one you are already
> using, I'm wondering what your reasoning for this is.
I can't speak for Chris or others. For us, there are just too many
broken packages in some distros, that render it difficult to fix. If
you stick with the "officially supported" bits, you have to live with
broken packages. If you install your own, the "helpful" package manager
will happily overwrite your updated package if you didn't use the
default packaging mechanism for the update. And if you did, it will
report a conflict/dependency purgatory for you to unwind.
This is why I tend to favor installing as minimal systems as possible if
you install them. Most distros like creating huge dependency radii for
their packages. Trying to remove some unneeded sound libraries recently
under SuSE caused the package manager to eventually wish to remove
glibc. Yeah, this could be a broken dependency list. Or it could be a
real dependency. Somehow.
Some distros don't try to get in your way so much, and actually let you
replace bits at a sub-system level. I have been pretty impressed with
Ubuntu(Debian) in this regard.
> Granted in theory it *should* be easy to change distro and it has
> getting easier as they have become more and more homogeneous over time
> but equally replacing the RedHat Kernel with one that supports XFS isn't
> the end of the world either.
Agreed, though all the other broken bits (Perl, et al), just make you
want to chuck the whole distro in favor of something more modern.
Usually we deliver our software in its own distinct namespace tree.
This allows us to not depend upon the distro issues. This way we fight
bugs, not distro problems. Well, most of the time.
> Are you choosing a distro which you can run un-modified and hence buy
> support for or do you genuinely believe it's easier to learn and support
Hmmm... I am not sure if Chris wants to "buy" support for the distro.
Last I checked the only two distros I knew of with any sort of HPC focus
are Scyld and Caos. Everything else is focused upon LAMP service and
related. Buying support from a distro provider who doesn't grasp how
the system is being used might not fly.
> a new distro than it is to install a new kernel on the current one?
> Personally I've always been of the opinion that you should pick a distro
> based on what you have experience of in-house and then modify that
> distro to meet your needs rather than the other way around.
If you spend weeks and weeks fighting broken bits after broken bits, it
gets a little tiring. I would say that a quick cost benefit analysis is
in order. If you can do it more quickly with less effort on another
distro, frankly, there is little reason why you shouldn't.
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax : +1 866 888 3112
cell : +1 734 612 4615
More information about the Beowulf