FYI: Current SPECfp landscape...

[Greg Lindahl]

On Thu, May 10, 2001 at 03:53:10PM -0400, Josip Loncaric wrote:

> The point is data compression: I can convey the first order
> approximation in one number, with minor (<5%) errors.

You can? So when the Athlon comes out with a faster memory system,
your answers break, or if the P4 gets a different chipset with a
slower memory system, your answer breaks. It also leads you to this
kind of mistake:

> Also, "per GHz" comparisons tell you something
> about a particular architecture.  The Athlon FPU is expected to
> outperform the P3 FPU by 4:3 per GHz, but on SPECfp2000 P4 benchmarks,
> this effect is *not* obvious.

... because you just ignored the effect of memory bandwidth, which is
growing as a factor in SPECfp over time. Not to mention the effect of
the ability of a vectorizing compiler to generate SSE2 instructions on
the particular code in SPEC2000fp, etc etc.

SPEC2000fp is a fine benchmark, if you look at the absolute results.
The minute you start computing derived numbers from that, you're
making assumptions and generalizations. If you want to predict the
probable SPEC2000fp of a new processor by looking at its clock and the
clock of a very similar existing chip, that's probably fine. But
comparing to a completely different chip?

-- g