[Beowulf] Pretty Big Data

[Perry E. Metzger]

On Mon, 25 Jan 2016 21:42:29 +0000 "Lux, Jim (337C)"
<james.p.lux at jpl.nasa.gov> wrote:
> I think the original thought is that for some set of problems, a
> brute force keep it all in RAM approach is as good, if not better,
> than anything more sophisticated, if you're starting to put
> development/test/etc costs.

It is true that the world has very powerful rocket engines
now. I suspect that using a powerful enough rocket engine, you could
move along an intermodal transport container down a highway without
needing any wheels, the engines being powerful enough to overcome the
friction of the roadway against the container. I am not sure,
however, that this is a good idea.

Regardless: any simple search structure you can name is as
easy to use as a linear array or list in a modern language.
Even in C++, the cost of using a reasonable fast-to-search
data structure from the STL is no worse (in terms of API complexity)
than using a vector and brute forcing your way through it. In other
languages, it is even easier.

So, there's really no point in not using a reasonable data structure
-- it does not save any development or test costs, is not observably
more complicated, and will save you many, many, many orders of
magnitude in run time if you're dealing with a large amount of data.

With that, I'll bow out of this.

Perry
-- 
Perry E. Metzger		perry at piermont.com