How to assess and report the performance of a stochastic algorithm on a benchmark problem: mean or best result on a number of runs?

Mauro Birattari, Marco Dorigo
2006 Optimization Letters  
Some authors claim that reporting the best result obtained by a stochastic algorithm in a number of runs is more meaningful than reporting some central statistic. In this short note, we analyze and refute the main argument brought in favor of this statement.
doi:10.1007/s11590-006-0011-8 fatcat:m23hyywbovca7fyylnes6t3uqa