[Beowulf] Storage memory
Ellis H. Wilson III
ellis at cse.psu.edu
Thu Oct 31 21:14:52 PDT 2013
On 10/31/2013 11:35 AM, John Hearns wrote:
> A concept well worth discussing.
> I think I've broached this topic a couple of times on this list - we are
> seeing ever more applications hungry for large amounts of RAM,
> whether or not that is just programmers dimensioning larger and larger
> arrays, or applications using finer and finer mesh sizes to
> capture detail of physical processes.
> But does that data have to be in main memory - I think 'storage memory'
> like this has a great future.
> Also add in the requirement of Big Data in-memory databases too.
If any of you are attending SC (and naturally, the corresponding
beobash), it may be of interest that I'll be giving a presentation on my
research, which is closely related to this query in many ways. If you
are interested, see the following for the abstract:
Regarding the article, here are a few thoughts regarding DIMM-based
non-volatile memory as that is another big push of the research group
I'm a part of. My current overarching thoughts on the topic:
1. Cards that size break a lot of JEDEC standards -- it won't be
accepted readily by most setups. It might "Just Work" in some cases,
but breaking standards is a bad thing to do IMHO.
2. Cards JEDEC size and thereabouts (in other words, normal RAM stick
size) will hold significantly /less/ than current best of breed SATA or
PCIe alternatives. Sure, you can throw a few more DIMMs in, but you
could do the same with regard to the SATA or even PCIe (up to a point).
So capacity will be a big problem, although I suppose if you are
solely trying to use this as plain ol' memory, comparing it against DRAM
might be a winning comparison. However, considering PCIe3 bandwidth is
comparable to at least DDR3 bandwidths in my price range, there isn't a
big benefit to NAND in DIMM on that front.
3. Considering latency, NAND in a DIMM is and will continue to be
limited by its own physical characteristics. This isn't going to get
better either, as more bits per cell are added and feature sizes
decrease. Future tech (PCM/STTRAM/MRAM/etc) may improve that -- I
cannot speak to that confidently in any sense at this time.
4. Any product that suggests it has solved the byte-addressability
problem with NAND is a hack. It's simply hiding the real
characteristics of NAND behind buffers and algorithms. Maybe big
buffers and complex algorithms, but at its core NAND is NOT
byte-addressable. If it were, we'd call it NOR.
5. If you have an application that absolutely needs to have all of its
data in memory, that screams "random access workload" to me at least.
Assuming a uniform distribution of accesses over the entire space,
particularly if you need to do a lot of writing, will not be a friendly
environment for NAND based DIMMs unless your records are block-sized,
which is probably few cases. So, excuse my own doubt about general
purpose NAND DIMMs taking over that market any time soon.
More information about the Beowulf