A Comparison of Test Case Prioritization Criteria for Software Product Lines

Ana B. Sanchez, Sergio Segura, Antonio Ruiz-Cortes
2014 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation  
Software Product Line (SPL) testing is challenging due to the potentially huge number of derivable products. To alleviate this problem, numerous contributions have been proposed to reduce the number of products to be tested while still having a good coverage. However, not much attention has been paid to the order in which the products are tested. Test case prioritization techniques reorder test cases to meet a certain performance goal. For instance, testers may wish to order their test cases in
more » ... order to detect faults as soon as possible which would translate in faster feedback and earlier fault correction. In this paper, we explore the applicability of test case prioritization techniques to SPL testing. We propose five di↵erent prioritization criteria based on common metrics of feature models and we compare their e↵ectiveness in increasing the rate of early fault detection, i.e. a measure of how quickly faults are detected. The results show that di↵erent orderings of the same SPL suite may lead to significant di↵erences in the rate of early fault detection. They also show that our approach may contribute to accelerate the detection of faults of SPL test suites based on combinatorial testing.
doi:10.1109/icst.2014.15 dblp:conf/icst/SanchezSC14 fatcat:qjs5dfszx5bnvf6zuv5mkia6rq