3,156 Hits in 5.9 sec

Relating Code Coverage, Mutation Score and Test Suite Reducibility to Defect Density

David Tengeri, Laszlo Vidacs, Arpad Beszedes, Judit Jasz, Gergo Balogh, Bela Vancsics, Tibor Gyimothy
2016 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)  
This paper reports on an experiment conducted on four open source systems' test suites to compare them from the viewpoints of code coverage, mutation score and test suite reducibility (the amount test  ...  We demonstrate that in some situations code coverage might be a sufficient indicator of the expected defect density, but mutation and reducibility are better in most of the cases.  ...  Analysis of Results In this study we used code coverage, mutation score and test reducibility as different indicators of test suite adequacy, which we aim to compare with defect density, an independent  ... 
doi:10.1109/icstw.2016.25 dblp:conf/icst/TengeriVBJBVG16 fatcat:lvdvw426n5fqtl62oevuaakdpi

On the effectiveness of unit tests in test-driven development

Ayse Tosun, Muzamil Ahmed, Burak Turhan, Natalia Juristo
2018 Proceedings of the 2018 International Conference on Software and System Process - ICSSP '18  
We also measure branch and method coverage of test suites to compare our results with the literature.  ...  Results: In terms of mutation score, we have found that the test cases written for a test-driven development task have a higher defect detection ability than test cases written for an incremental test-last  ...  Mutation score indicator. Mutation testing is a way to measure the ability of a test case to catch defects in the code [31] .  ... 
doi:10.1145/3202710.3203153 dblp:conf/ispw/TosunATJ18 fatcat:qzcrs6mbffbgdm72zys5gdce7i

State Coverage: Software Validation Metrics beyond Code Coverage [chapter]

Dries Vanoverberghe, Jonathan de Halleux, Nikolai Tillmann, Frank Piessens
2012 Lecture Notes in Computer Science  
Currently, testing is still the most important approach to reduce the amount of software defects.  ...  Currently, testing is still the most important approach to reduce the amount of software defects.  ...  Currently, testing is still the most important approach to reduce the amount of software defects.  ... 
doi:10.1007/978-3-642-27660-6_44 fatcat:vhfn7ejb2jhyxkwcqafizb4sq4

Characterizing High-Quality Test Methods: A First Empirical Study [article]

Victor Veloso, Andre Hora
2022 arXiv   pre-print
To assess the quality of a test suite, one can rely on mutation testing, which computes whether the overall test cases are adequately exercising the covered lines.  ...  In this paper, we propose an empirical study to assess the quality of test methods by relying on mutation testing at the method level.  ...  Code coverage and mutation score typically target the overall test suite effectiveness [6, 8, 9, 26] . However, this high level of granularity may overshadow test quality [12] .  ... 
arXiv:2203.12085v2 fatcat:ffjbie27zjahhgue7egsj3hwe4

Can testedness be effectively measured?

Iftekhar Ahmed, Rahul Gopinath, Caius Brindescu, Alex Groce, Carlos Jensen
2016 Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering - FSE 2016  
The first measure is statement coverage, the simplest and best-known code coverage measure. The second measure is mutation score, a supposedly more powerful, though expensive, measure.  ...  Test the least-tested code, and stop when all code is well-tested, is a reasonable answer.  ...  [21] found that statement, out of branch, and path coverages, best correlated with mutation score, and hence may best predict defect density, in a study that compared suites and mutation scores across  ... 
doi:10.1145/2950290.2950324 dblp:conf/sigsoft/AhmedGBGJ16 fatcat:m7k6o5dykngjhjqphjahkrmrxe

The Relation of Test-Related Factors to Software Quality: A Case Study on Apache Systems

Fabiano Pecorelli, Fabio Palomba, Andrea De Lucia
2021 Empirical Software Engineering  
Recent studies have shown that test-related factors (e.g., code coverage) can be reliable predictors of software code quality, as measured by post-release defects.  ...  While these studies provided initial compelling evidence on the relation between tests and post-release defects, they considered different test-related factors separately: as a consequence, there is still  ...  p <0.1 Acknowledgments The authors would like to sincerely thank the Associate Editor and anonymous Reviewers for the insightful comments and feedback provided during the review process.  ... 
doi:10.1007/s10664-020-09891-y fatcat:psfducgepfatjovzinfjchifpa

On guiding the augmentation of an automated test suite via mutation analysis

Ben H. Smith, Laurie Williams
2008 Empirical Software Engineering  
Mutation testing has traditionally been used as a defect injection technique to assess the effectiveness of a test suite as represented by a "mutation score."  ...  Our results indicate that a focused effort on increasing mutation score leads to a corresponding increase in line and branch coverage to the point that line coverage, branch coverage and mutation score  ...  Acknowledgements We would like to thank the North Carolina State University RealSearch group for their helpful comments on the paper.  ... 
doi:10.1007/s10664-008-9083-7 fatcat:wmmvtvhxxvggtmd7qfzb2h4hpi

Characterization Of Open-Source Applications And Test Suites

Sarojini Balasubramanian
2018 Zenodo  
The study also found that there is a positive correlation between the mutation score and the coverage score.  ...  We found that increasing the coverage of the test suites for applications with at least 500 test cases can make the test suites more costly. The correlation coeffieicent obtained was 0.543.  ...  Code with higher coverage tends to have fewer field defects making it a practical technique to measure test suite quality [27] .  ... 
doi:10.5281/zenodo.1327072 fatcat:ejw3gd6yqjh2xc7324urnxssbi

Lightweight Lexical Test Prioritization for Immediate Feedback

Toni Mattis, Robert Hirschfeld
2020 The Art, Science, and Engineering of Programming  
With growing and longer running test suites, however, feedback is obtained less frequently and lags behind program changes.  ...  The canonical approach relies on coverage data and prioritizes tests that cover the changed region, but obtaining and updating coverage data is costly.  ...  Acknowledgements We thank Falco Dürsch, who through his work on history-based test prioritization contributed to our understanding of test prioritization in general; the HPI Research School for Service-Oriented  ... 
doi:10.22152/ fatcat:ogb5somknjfjzdmh42wknhbura

Automated assessment of students' testing skills for improving correctness of their code

Zalia Shams
2013 Proceedings of the 2013 companion publication for conference on Systems, programming, & applications: software for humanity - SPLASH '13  
Code coverage measures are typically used in practice, but that approach does not assess how much of the expected behavior is checked by the tests and sometimes, overestimates the true quality of the tests  ...  We propose novel and practical solutions to apply all-pairs testing and mutation analysis of student-written tests, especially in the context of classroom grading tools.  ...  This mechanism gives students a greater realization of the density of bugs in their code and their ability to write tests that find defects in others' solutions.  ... 
doi:10.1145/2508075.2508078 dblp:conf/oopsla/Shams13 fatcat:mhoy4a3jpff7tkpslg2dgjqtwe

ConPredictor: Concurrency Defect Prediction in Real-World Applications

Tingting Yu, Wei Wen, Xue Han, Jane Hayes
2018 IEEE Transactions on Software Engineering  
Software defect prediction has been used to help developers find faults and prioritize their testing efforts.  ...  In this paper, we present ConPredictor, an approach to predict defects specific to concurrent programs by combining both static and dynamic program metrics.  ...  ACKNOWLEDGMENTS This work was supported in part by NSF grants CCF-1464032, CCF-1652149, and CCF-1511117.  ... 
doi:10.1109/tse.2018.2791521 fatcat:3ljq4x7s2za6jkmdumuy66lz54

Software mutational robustness

Eric Schulte, Zachary P. Fry, Ethan Fast, Westley Weimer, Stephanie Forrest
2013 Genetic Programming and Evolvable Machines  
Test cases are used to measure program behavior and mutation operators are taken from  ...  We apply these concepts to software, defining mutational robustness to be the fraction of random mutations to program code that leave a program's behavior unchanged.  ...  Acknowledgments The authors would like to thank Lauren Ancel Meyers, William B. Langdon and Peter Schuster for their thoughtful comments.  ... 
doi:10.1007/s10710-013-9195-8 fatcat:kaqrckbi2ng5jebkeskhyefxqq

Empirically studying the role of selection operators duringsearch-based test suite prioritization

Alexander P. Conrad, Robert S. Roos, Gregory M. Kapfhammer
2010 Proceedings of the 12th annual conference on Genetic and evolutionary computation - GECCO '10  
This paper presents a genetic algorithm-based test prioritization method that employs a wide variety of mutation, crossover, selection, and transformation operators to reorder a test suite.  ...  prioritizer is superior to random search and hill climbing and thus suitable for many regression testing environments.  ...  a test suite with a high coverage effectiveness score.  ... 
doi:10.1145/1830483.1830735 dblp:conf/gecco/ConradRK10 fatcat:xpspwy62hfhgpajrkam3ogerm4

Analysis of Mutation Testing Tools in Aspect Oriented Software Engineering

Vineeta Vineeta, Preeti Lochab, Abhishek Singhal
2013 International Journal of Computer Applications  
Mutation testing measures the adequacy of the test suite by seeding artificial defects i.e. mutants in the program.  ...  Mutation score When we execute a mutant using a test suite, we may have any of the following outcomes: The result of the program is affected by the change and any test case of the test suite detects it  ...  It is also known as Mutation Analysis and Program Mutation. A test suite that does not detect and reject the mutated code is considered defective.  ... 
doi:10.5120/9965-4618 fatcat:ea37q6zrtbhzjnd7tlbqmdfuea

Unit Test using Test-Driven Development Approach to Support Reusability

Myint Myint Moe
2019 International Journal of Trend in Scientific Research and Development  
In TDD, the tests are written before the code is implemented as test first. The number of defects reduced when automated unit tests are written iteratively similar to test driven development.  ...  Writing test first can assure the correctness of the code and thus helping developer gain better understanding of the software requirements which leads to fewer defects and less debugging time.  ...  Author also measures branch and method coverage of test suites to compare author's results with the literature.  ... 
doi:10.31142/ijtsrd21731 fatcat:lswz5v2gnndgtbescluqmjo5bi
« Previous Showing results 1 — 15 out of 3,156 results