[Beowulf] Rackable / SGI

[Joe Landman]

Greg Lindahl wrote:
> On Fri, Apr 03, 2009 at 08:15:06PM -0400, Joe Landman wrote:
> 
>> It has been, realistically, the only serious choice for big/fast  
>> data systems on Linux for quite some time.
> 
> Most big/fast data systems don't have big filesystems. In HPC, the
> biggest use something like Lustre; in business, the biggest use
> something like BigTable/GFS. In both cases the underlying filesystems
> don't have to be that big.

We see everything from a few TB through a few PB.  Not every cluster FS 
we see is Lustre.  We have had some interesting conversations about this 
with customers in the last several days.

> Now for what you really meant ("big filesystems"), that was pretty
> true when XFS first came out. But Emacs (oops, I meant ext2) was
> improved upon subsequently, and the remaining differences are more a
> lifestyle choice than a clear technical choice.

Hmmm... There were people who knew they needed it, people who didn't 
think they needed it (but did), and those who didn't need it at all. 
The latter group didn't deal with GB sized data sets (or larger), and 
often ran the same thing they ran on their desktop as they perceived it 
to be "good enough".

> 
>> Ext3 has some serious  
>> performance limits due to its journaling design.  Never mind its other  
>> issues.  There was mention of this in this past week's LWN.net.
> 
> If you use the default desktop config, yes. This is a common situation
> in lifestyle arguments: "performance would be poor if you use product
> X like an idiot." It's an ease-of-use problem.

Heh ... as someone who performance tunes the systems and actively 
compares the performance ... well ...

... if your use case is rm -rf /dir, sure ext3 is better.  If you have 
to move and interact with lots of data (GB+ sized loads) on a regular 
basis ... and you believe that ext3 is still your friend when you tune 
it "optimally", then that, indeed, is a lifestyle choice.  Not unlike 
living in a shack out in the woods without a bathroom.  Real men use 
ext3 and use leaves (or don't wipe) ... or something like that.

[sarcasm /]

[...]

>> Zfs is not the revealed word of some deity, in file systems.  This mind  
>> set is painful to deal with, and often winds up with people having  
>> *very* unrealistic expectations of what it is, what it can do, and how  
>> it performs.
> 
> Yeah, it almost reminds me of when XFS came out. (Did I say that out
> loud?)

You certainly did.  Not true, but you are of course welcome to your 
opinion.  XFS isn't great for everything.  We don't use it for 
everything.  Just where large block sequential performance matters, and 
size limits can be a problem.

> You're probably a vi user :-)

Nope, not my primary editor, but I will use it over slow connections.  I 
prefer nedit (scary to think I used a way early version of it to write a 
thesis), pico/nano.  vim/gvim doesn't suck (vi straight does).

I have never learned the operating system known as emacs.  Its simply 
too complex for mere mortals.  Sort of like sendmail config sequences.



-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
        http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615