Java Unit Testing Tool Competition

Xavier Devroey, Sebastiano Panichella, Alessio Gambi
2020 Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops  
We report on the results of the eighth edition of the Java unit testing tool competition. This year, two tools, EvoSuite and Randoop, were executed on a benchmark with (i) new classes under test, selected from open-source software projects, and (ii) the set of classes from one project considered in the previous edition. We relied on an updated infrastructure for the execution of the different tools and the subsequent coverage and mutation analysis based on Docker containers. We considered two
more » ... fferent time budgets for test case generation: one an three minutes. This paper describes our methodology and statistical analysis of the results, presents the results achieved by the contestant tools and highlights the challenges we faced during the competition. CCS CONCEPTS • Software and its engineering → Search-based software engineering; Automatic programming; Software testing and debugging.
doi:10.1145/3387940.3392265 dblp:conf/icse/DevroeyPG20 fatcat:5yg2wyk2z5cdtmlm2w63u3vtnu