Automated bug assignment: Ensemble-based machine learning in large scale industrial contexts

Leif Jonsson, Markus Borg, David Broman, Kristian Sandahl, Sigrid Eldh, Per Runeson
2015 Empirical Software Engineering  
Context: Bug report assignment is an important part of software maintenance. In particular, incorrect assignments of bug reports to development teams can be very expensive in large software development projects. Several studies propose automating bug assignment techniques using machine learning in open source software contexts, but no study exists for large-scale proprietary projects in industry. Objective: The goal of this study is to evaluate automated bug assignment techniques that are based
more » ... on machine learning classication. In particular, we study the state-of-the-art ensemble learner Stacked Generalization (SG) that combines several classiers. Method: We collect more than 50,000 bug reports from ve development projects from two companies in dierent domains. We implement automated bug assignment 2 Leif Jonsson et al. and evaluate the performance in a set of controlled experiments. Results: We show that SG scales to large scale industrial application and that it outperforms the use of individual classiers for bug assignment, reaching prediction accuracies from 50% to 90% when large training sets are used. In addition, we show how old training data can decrease the prediction accuracy of bug assignment. Conclusions: We advice industry to use SG for bug assignment in proprietary contexts, using at least 2,000 bug reports for training. Finally, we highlight the importance of not solely relying on results from cross-validation when evaluating automated bug assignment.
doi:10.1007/s10664-015-9401-9 fatcat:oemqchszfvdb7eus6w2zgmumoi