[Beowulf] BIG 'ram' using SSDs - was single machine with 500 GB of RAM

[Ellis H. Wilson III]

On 01/09/2013 02:24 PM, Vincent Diepeveen wrote:
>
> On Jan 9, 2013, at 8:02 PM, Ellis H. Wilson III wrote:
>>
>> Yea, it's computer science, and I'd love to see you try to toss 16
>> crappy SSDs in a box with a crappy RAID controller and get this easy
>> 2.7GB/s random accesses you are touting.  Not going to happen.
>>
>
> If you had done effort reading what i wrote - i had said that the

It was commensurate with what you put into the writing process.

> random latency is
> under 70 us (achieved) at SSD's, so doubting that is kind of
> nonsense. it's what you ACHIEVE.

Yea.  You're right.  Constantly spitting crazy nonsense:
http://www.cse.psu.edu/~mqj5086/uploads/Main/MJ_PAQ.pdf

> You mix that up now as that you can get with random reads a speed of
> 2.7 GB/s,

No, my assumption is that if the poster says he needs tons of RAM, his 
application probably has a random-access pattern.  You harping on 
sequential throughput is not constructive to the conversation, unless 
you have identified the workload for the application is indeed 
sequential via tracing.

> Writes really isn't the problem at hardware that has lots of parallel
> channels.

They can be easier, as the publication above points out, but they still 
are bound by the nature of MLC being slower at the device-level for 
writes than reads.  This is particularly for write-in-place and 
random-writes, which I suspect this workload may have enough of to worry 
about.

> Maybe you should do some effort to read better. You would also have
> i started a new subject here. Namely not being busy with 500GB.
>
> 500GB is total peanuts. Cheapskate oldie 4 socket AMD box with 500GB
> and done.
> No need to discuss even, as intel is too expensive simply at that
> range, not to mention IBM.

Oh, sorry, I assumed we were still having a constructive conversation 
about the posters topic to help him achieve his goal for his application 
rather than merely thread-stealing and upping the storage size by 10X 
because "500GB is total peanuts."  I'll remember to keep my measuring 
stick handy next time I think about trying to help somebody build a 
machine specific to their problem so I don't accidentally stumble into 
"peanut" territory.

Best,

ellis