[Beowulf] SSD caching for parallel filesystems

[Vincent Diepeveen]

On Feb 10, 2013, at 3:06 PM, Ellis H. Wilson III wrote:

> On 02/10/13 08:40, Vincent Diepeveen wrote:
>>
>> On Feb 10, 2013, at 2:09 PM, Ellis H. Wilson III wrote:
>>
>>> On 02/10/13 04:41, Vincent Diepeveen wrote:
>>>> SSD's are not about bandwidth, they're about latency.
>>>
>>> This is a bit aggressive of a vantage point -- let's tone it back:
>>> "SSD's aren't always the cheapest way to achieve bandwidth, but  
>>> they are
>>> critical for latency-sensitive applications that are too large  
>>> for main
>>> memory."
>>
>> SSD's are never the cheapest way to achieve bandwidth and never  
>> will be.
>
> Ok.  These kinds of statements are my favorite of yours :D.

>
>> Find me an application that needs big bandwidth and doesn't need  
>> massive
>> storage.
>
> Out-of-core ab-initio nuclear simulations.  Please see the paper I
> referenced a few emails ago.  These folks had a lot of data, certainly
> more than could fit in DRAM reasonably, but not so much that they  
> needed
> tons and tons of HDDs to hold it.  Let's call it "medium-data."  Oddly
> enough, SSDs fall right in the "middle" between DRAM and HDD latencies
> and capacity.  Weird right?

Not really - they're closer to RAM of course technologywise.

Just a lot of generations older :)

Price also is that of slow old RAM piled up into a casing :)

>
>> So any SSD solution that's *not* used for latency sensitive  
>> workloads,
>> it needs thousands of
>> dollars worth of SSD's.
>> In such case plain old harddrive technology that's at buy in price  
>> right
>> now $35 for a 2 TB disk
>> (if you buy in a lot, that's the actual buy in price for big shops  
>> and
>> you nor i get them for that price
>> of course), or $17.5 a terabyte, that's unbeatable in performance for
>> storage and bandwidth.
>
> Right.  So since nobody I know can get them at that price, let's
> continue talking about them in terms of their real price: 150-200
> dollars.  I'm not going to have a discussion about imaginary pricing
> nobody I know of has access to.

pardon me?

3 TB raid disks delivering 200 MB/s if you just buy one are 120 euro  
here.

Yet i remember some bragging recently on storage and how many of such  
disks some HPC centers
use, so you get them dirt cheap then and with some negotiations you  
really get that down bigtime
of course.

Most HPC centers here buy in by the many petabytes already for over  
10 years.

For the SSD's on other hand - you never will buy enough for your HPC  
center i guess,
so you always pay the full price.

>
>> We speak about a sustained 200MB/s for dirt cheap RAID harddrives  
>> here.
>> Put 16 of them in a raid partition and
>> you can get more than you can deliver over the network from the file
>> server and more than your motherboard can effectively
>> handle a second.
>>
>> We speak about a buy in price of total peanuts for 16 harddrives  
>> here,
>> and the same storage in SSD is
>> worth a total fortune.
>
> Please elaborate on total peanuts versus total fortune, and feel  
> free to
> use real numbers from NewEgg or other retail figures.  Feel free to
> exclude eBay from your search.

How many machines are actually to your availability there and how  
much storage in total is
available in the location you sit in running the simulations?

Because then it's easier to determine a price.

>
>> So using SSD's is just for latency. Anyone not using them for that i
>> would never hire.
>
> Oh well.  I was never much into chess anyhow, ;).
>
> Best,
>
> ellis
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
> Computing
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf