Automated Detection of Performance Regressions: The Mono Experience

T. Kalibera, L. Bulej, P. Tuma
13th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems  
Engineering a large software project involves tracking the impact of development and maintenance changes on the software performance. An approach for tracking the impact is regression benchmarking, which involves automated benchmarking and evaluation of performance at regular intervals. Regression benchmarking must tackle the nondeterminism inherent to contemporary computer systems and execution environments and the impact of the nondeterminism on the results. On the example of a fully
more » ... of a fully automated regression benchmarking environment for the Mono opensource project, we show how the problems associated with nondeterminism can be tackled using statistical methods.
doi:10.1109/mascots.2005.18 dblp:conf/mascots/KaliberaBT05 fatcat:koqm6tydcvb5lj5bnlrkg72u2y