[Beowulf] PetaBytes on a budget, take 2

[Joe Landman]

On 07/22/2011 12:33 AM, Mark Hahn wrote:
>> Either way, I think if someone were to foolishly just toss together
>>> 100TB of data into a box they would have a hell of a time getting
>> anywhere near even 10% of the theoretical max performance-wise.
>
> storage isn't about performance any more.  ok, hyperbole, a little.
> but even a cheap disk does>  100 MB/s, and in all honesty, there are
> not tons of people looking for bandwidth more than a small multiplier
> of that.  sure, a QDR fileserver wants more than a couple disks,

With all due respect, I beg to differ.

The bigger you make your storage, the larger the pipes in you need, and 
the larger the pipes to the storage you need, lest you decide that tape 
is really cheaper after all.

Tape does 100MB/s these days.  And the media is relatively cheap 
(compared to some HD).  If you don't care about access performance under 
load, you really can't beat its economics.

More to the point, you need a really balanced architecture in terms of 
bandwidth.  I think USB3 could be very interesting for small arrays, and 
pretty much expect to start seeing some as block targets pretty soon.  I 
don't see enough aggregated USB3 ports together in a single machine to 
make this terribly interesting as a large scale storage medium, but it 
is a possible route.

They are interesting boxen.  We often ask customers if they'd consider 
non-enterprise drives.  Failure rates similar to the enterprise as it 
turns out, modulo some ridiculous drive products.  Most say no.  Those 
who say yes don't see enhanced failure rates.

> and if you're an iops-head, you're going flash anyway.

This is more recent than you might have guessed ... at least outside of 
academia.  We should have a fun machine to talk about next week, and 
show some benchies on.



-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615