[Beowulf] GPFS on Linux (x86)

[Craig Tierney]

laytonjb at charter.net wrote:
> Craig,
> 
>>> http://www.clustermonkey.net//content/view/31/28/
>> The website doesn't have any real-world experience.  I would
>> like to know things like 'Filesystem X has much better
>> meta-data performance than Filesystem Y', or 'Don't try using
>> netcdf files on Filesystem Z, because the performance will
>> be awful'.
>>
>> Discussions about real-world use will help users get through
>> the marketing fluff to find out what is really going on.  Not
>> everyone has a chance to test every single filesystem before
>> choosing one.
> 
> Since I'm the author of the article let me say - I agree with you.
> But let me also say that installing each of the approriate file systems would
> be difficult and very time consuming. I would love to do (file systems - if
> you are listening...) but I also need some hardware to do it. Any volunteers?
> More over, the file systems have their preferred hardware configurations for
> best performance. So a single hardware configuration may not be the most
> appropriate for all of the file systems (in addition, Panasas is a combined
> hardware/software solution so that make the comparison a bit more difficult).
> 
> The intent of the article is a "survey." Just a quick review of what's available,
> some general "features" of the file system, perhaps how you configure it, etc.
> It's not intended as a review, since, as you have pointed out, there was no
> testing done.
> 
> BTW - I'm working on a revision of the article. I hope to update it a bit and
> provide more technical details on the file systems.
> 
> Also, I have real world experience with several of the file systems, but  I haven't
> done any comparisons on performance. I was just curious to experiment with
> the file systems.
> 
> Finally, don't let my limitations hold anyone back. if anyone wants to do some
> reviews/comparisons/etc. please feel free to do so and write up the results for
> ClusterMonkey (shameless plug there). If you need help with the comparisons,
> I'm glad to help. I'll even help write the article if you would like.
> 
> Jeff
> 

Jeff,

I hope you know I wasn't trying to be negative about your article.  I 
thought it provided a great summary.

The important thing to remember about this discussion is look at all
of the options we have.  Some are open-source, and some are commercial. 
  Go back 3-5 years and look at what filesystems were available for 
Linux.  If they existed and even worked, they didn't scale well or were 
targeted
for a very specific purpose.  Now, all of the filesystem options are 
much more general-purpose. There are some really amazing things we can 
start to do with storage and filesystems that weren't possible or 
required huge development teams and huge budgets.  Things are close to 
'just working' which is what most people want anyway.

Craig