[Beowulf] MPI application benchmarks

[Toon Knapen]

Martin Siegert wrote:
>> Why do'nt you make a list of multiple-choice questions in a style as 
>> described above and ask your users to fill that in. This solves also the 
>> 'weighting factor' because the users that respond to your question 
>> _care_ about the machine being suitable while the others care less.
> 
> Several reasons why this does not work quite the way we would like:
> - it is surprising (or not ...) how many users simply do not know
>   how to characterize their application. The only way is to get a
>   copy of those applications that chew up most of the walltime,
>   compile them, try to understand what the application is doing,
>   and then classify it yourself. Takes a lot of time ... unless
>   it is a well known application like gromacs.


Are your users using commercial apps or home-brewn apps?

If your users use home-brewn stuff, they (at least the coder) should 
know how to charaterize the code.

If your users use commercial apps, ask the vendor of the app. They 
should be able to consult you in what should be the best hardware for 
their app (at least, that's what we do)


> - we start the work on the benchmark now. The RFP will be issued
>   many months down the road. The equipment will be purchased many
>   more months down the road, ... At the time when users get on the
>   facilities the users have change, their applications have changed,
>   etc., etc. Thus, "our own applications" today are not (necessarily)
>   relevant tomorrow when the equipment is purchased.
> 


Carefully engineered apps that try to get every bit of performance out 
of the system do not evolve (in terms of requirements on the system) 
that fast. So for the apps that do care about performance, the 
requirements do not change fast over time.