[Beowulf] SSD caching for parallel filesystems

[Joe Landman]

On 02/11/2013 06:26 PM, Vincent Diepeveen wrote:
> You make a joke out of it,
>
> Yet SSD's you never buy in in huge quantities, whereas for any self

The great thing about  gross overarching generalizations is that they 
tend to be incorrect.  They are sort of a recursive joke.

First, define "huge".

Is 10 huge?  100?  1000?

What is huge?

> respecting HPC practicing organisation or company,

Ahh ... I have to ask Vincent, do you run such a "self respecting HPC 
practicing organisation or company " ?  I am curious as to where this 
absolute knowledge comes from.


> they buy in massive harddrive storage, so they *can* get every single
> harddrive for the prices quoted, which is between $10 and $20 a terabyte
> right now, depending upon which quality harddrive, reliability,
> bandwidth and manufacturer you buy.

$10 - $20 /TB?  For desktop drives, certainly, you can get the slow 
(5400RPM) cheap ones for about $40/TB or so (+/- some).

>
> If such harddrive raidt doesn't deliver enough bandwidth - requiring
> you to also use SSD's - you're doing something completely wrong as a
> HPC organisation.

... and again, absolutes ... which are .... oh so wonderfully wrong ... 
but lets keep going.

>
> The RAID controllers also are on the pci-e and both will be
> delivering 3GB/s usable user data to your cluster no matter how many
> harddrives
> or SSD's you put in parallel i bet at a single pci-e controller.
>
> Though it would be fun if someone tested it whether you can get a
> larger sustained bandwidth than that.

If you know what you are doing, 3GB/s is so ... several years ago. c.f.  
http://scalability.org/?p=3157

Wait, you want faster?  How about this? http://scalability.org/?p=5779

>
> What we know for sure is that the SSD array is going to be really
> expensive.

So far, in this post, these are the first words that I would have to 
agree with, for various values of the word "really".

>
> In short SSD's are only interesting for latency and caching.

... and back to absolutes.  Wrong ones, but absolutes.


So at some point, when its all ok to talk publicly about what we are 
doing, you'll see a pretty good refutation of, well, almost everything 
that was said in this post.

And yes, my company is very self respecting.



-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615