[Beowulf] Storage memory

[Ellis H. Wilson III]

On 10/31/2013 11:35 AM, John Hearns wrote:
> http://www.theregister.co.uk/2013/10/31/dimm_outlook_for_storage_memory/
> A concept well worth discussing.
> I think I've broached this topic a couple of times on this list - we are
> seeing ever more applications hungry for large amounts of RAM,
> whether or not that is just programmers dimensioning larger and larger
> arrays, or applications using finer and finer mesh sizes to
> capture detail of physical processes.
> But does that data have to be in main memory - I think 'storage memory'
> like this has a great future.
> Also add in the requirement of Big Data in-memory databases too.

<shameless plug>
If any of you are attending SC (and naturally, the corresponding 
beobash), it may be of interest that I'll be giving a presentation on my 
research, which is closely related to this query in many ways.  If you 
are interested, see the following for the abstract:
http://sc13.supercomputing.org/schedule/event_detail.php?evid=pap415
</shameless plug>

Regarding the article, here are a few thoughts regarding DIMM-based 
non-volatile memory as that is another big push of the research group 
I'm a part of.  My current overarching thoughts on the topic:

1. Cards that size break a lot of JEDEC standards -- it won't be 
accepted readily by most setups.  It might "Just Work" in some cases, 
but breaking standards is a bad thing to do IMHO.
2. Cards JEDEC size and thereabouts (in other words, normal RAM stick 
size) will hold significantly /less/ than current best of breed SATA or 
PCIe alternatives.  Sure, you can throw a few more DIMMs in, but you 
could do the same with regard to the SATA or even PCIe (up to a point). 
  So capacity will be a big problem, although I suppose if you are 
solely trying to use this as plain ol' memory, comparing it against DRAM 
might be a winning comparison.  However, considering PCIe3 bandwidth is 
comparable to at least DDR3 bandwidths in my price range, there isn't a 
big benefit to NAND in DIMM on that front.
3. Considering latency, NAND in a DIMM is and will continue to be 
limited by its own physical characteristics.  This isn't going to get 
better either, as more bits per cell are added and feature sizes 
decrease.  Future tech (PCM/STTRAM/MRAM/etc) may improve that -- I 
cannot speak to that confidently in any sense at this time.
4. Any product that suggests it has solved the byte-addressability 
problem with NAND is a hack.  It's simply hiding the real 
characteristics of NAND behind buffers and algorithms.  Maybe big 
buffers and complex algorithms, but at its core NAND is NOT 
byte-addressable.  If it were, we'd call it NOR.
5. If you have an application that absolutely needs to have all of its 
data in memory, that screams "random access workload" to me at least. 
Assuming a uniform distribution of accesses over the entire space, 
particularly if you need to do a lot of writing, will not be a friendly 
environment for NAND based DIMMs unless your records are block-sized, 
which is probably few cases.  So, excuse my own doubt about general 
purpose NAND DIMMs taking over that market any time soon.

Best,

ellis