[Beowulf] Checkpointing using flash

[Ellis H. Wilson III]

On 09/24/2012 10:56 AM, Lux, Jim (337C) wrote:
> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On Behalf Of Andrew Holway
> Sent: Monday, September 24, 2012 3:59 AM
> To: Eugen Leitl
> Cc: beowulf at beowulf.org
> Subject: Re: [Beowulf] Checkpointing using flash
>
>> Of course the physical modelers won't bat an eyelash, but the common
>> programmer who still tries to figure out this multithreading thing
>> will be out to lunch.
>
> Whenever you push a problem to from hardware software you exponentially increase the cost of solving that problem
>
>
> Because software people are more expensive?

Haha, I doubt it -- probably the opposite in terms of development cost. 
  Which is why I question the original statement on the grounds that 
"cost" isn't well defined.  Maybe the costs just performance-wise, but 
that's not even clear to me when we consider things at huge scales.

I think the MapReduce framework actually makes a good case for 
(admittedly non-general, fairly sequential workloads) the ability for 
software to cheaply and at reasonable performance scale with added 
hardware.  Don't expect to do any real physics on MR of course, but for 
huge data crunches it is quite nice.  A /totally/ general framework that 
scales on a number of platforms is one of those cure-alls we aren't 
likely to see for a decade or three.  Just my pessimistic perspective 
though :D.

Best,

ellis