[Beowulf] SPEC CPU 2006 released

[Vincent Diepeveen]

The programs inside spec are open source.

For example 458.sjeng is the free sjeng 11.2 version,
which we gave a 150MB hashtable and fixed a few old bugs which the
compiler guys noticed past years when testrunning specint2006.

Lucky there were more bugs in compilers than in the code to be fixed.

It's simply nearly impossible to license for example my diep's source code 
to
spec. Not because i would mind those compiler guys have my diep's source 
code,
as it will only have a positive effect that their compiler gets fast for my 
code.

But competitors will be able to simply buy a cheap spec license and see my 
source code.

As a result only non-commercial code will show up in spec.

That's the only disadvantage of spec, and it is a non-solvable problem in 
gametree search.

So apart from this single problem, and the 2 year unnecessary delay to 
create the testsuite,
what they do is pretty good idea in terms of benchmarking.

Someone without ties to a company must make a benchmark for systems, and 
let's face it.
95% of all persons on this list are totally subjective, some because they 
are paid by some company
and therefore will only see that company or one of its business partners as 
best, and again
others are still believing too much in the past where the hardware field 
advances quickly.

So someone must do objective testing with software of what is faster.

Then you get a compiler problem. So you need source code.

In all that spec is doing a reasonable good job compared to others.

Find me 1 site that 'tests' hardware that's objective. Spec is the best 
compromise.


Vincent


----- Original Message ----- 
From: "Geoff Jacobs" <gdjacobs at gmail.com>
To: <beowulf at beowulf.org>
Sent: Friday, August 25, 2006 9:01 PM
Subject: Re: [Beowulf] SPEC CPU 2006 released


> Mark Hahn wrote:
>>> The new SPEC cpu suite has been released, and initial results
>>> are posted to:
>>
>> is this one still stupidly not open-source?
>
> I don't think SPEC has any open source benchmarks.
>
> As far as I can tell, SPEC groups derive income by licensing their
> benchmarks to integrators and such (although I know SpecViewPerf is free
> to use). I would think they also have issues with proprietary 3rd party
> software within their benchmarks.
>
> I think they could be supported in the same manner as OSDL, but they are
> not, so no open source from SPEC.
>
> -- 
> Geoffrey D. Jacobs
>
> Go to the Chinese Restaurant,
> Order the Special
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf
>