[Beowulf] Accelerator for data compressing

[Nifty niftyompi Mitch]

On Fri, Oct 03, 2008 at 12:58:16PM -0700, Greg Lindahl wrote:
> On Fri, Oct 03, 2008 at 02:17:52AM -0700, Bill Broadley wrote:
> 
> > Er, that makes no sense to me.  You aren't going to end up with a smaller 
> > file by encoding a file less efficiently.
> 
> I often find that floating-point data doesn't compress much, but that
> ASCII representations of the same data compresses to a file smaller
> than the binary one. This is with gzip/bzip2, so we're talking
> dictionary-based compression -- and it's easy to imagine why this
> might be true. I've never seen any float-point-oriented compressor
> like the ones specialized for photos (jpeg, etc.)

Floating point numbers not compressing makes sense sense to me.
The noise of the LSB and layout would mask redunancy in the exponent and other
fields.  Since most compression tools are byte (character and string)
oriented in their design the ASCII version could expose the right stuff.

It does point out a gap in the compression tool suite that I think the
hard data folks might look at.....  interesting stuff. 


-- 
	T o m  M i t c h e l l 
	Found me a new hat, now what?