Filters








59,292 Hits in 6.0 sec

Assertions are strongly correlated with test suite effectiveness

Yucheng Zhang, Ali Mesbah
2015 Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering - ESEC/FSE 2015  
We find that the number of assertions in a test suite strongly correlates with its effectiveness, and this factor directly influences the relationship between test suite size and effectiveness.  ...  Our results also indicate that assertion coverage is strongly correlated with effectiveness and different types of assertions can influence the effectiveness of their containing test suites.  ...  Finding 4: Our results suggest that, assertion coverage is very strongly correlated with test suite effectiveness.  ... 
doi:10.1145/2786805.2786858 dblp:conf/sigsoft/ZhangM15 fatcat:6jbkvjucofhcrhtbgnfy7pzrra

Empirical Evaluation of Test Coverage for Functional Programs

Yufeng Cheng, Meng Wang, Yingfei Xiong, Dan Hao, Lu Zhang
2016 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST)  
The correlation between test coverage and test effectiveness is important to justify the use of coverage in practice.  ...  In this paper we report the first empirical study on the correlation between test coverage and test effectiveness on functional programs.  ...  The only exception is Finding 4, where Zhang and Mesbah found that the count of assertions is strongly correlated with test effectiveness on imperative programs.  ... 
doi:10.1109/icst.2016.8 dblp:conf/icst/ChengWXH016 fatcat:spznkoxmrbbw3pblwfmh2v2rrm

Does automated white-box test generation really help software testers?

Gordon Fraser, Matt Staats, Phil McMinn, Andrea Arcuri, Frank Padberg
2013 Proceedings of the 2013 International Symposium on Software Testing and Analysis - ISSTA 2013  
To investigate this issue, we performed a controlled experiment comparing a total of 49 subjects split between writing tests manually and writing tests with the aid of an automated unit test generation  ...  In the absence of a specification, a common assumption is that these tests relieve a developer of most of the work, as the act of testing is reduced to checking the results of the tests.  ...  Thus as EVOSUITE derived test suites evolve to become more effective, extraneous tests/assertions are dropped and replaced with tests using a smaller number of assertions.  ... 
doi:10.1145/2483760.2483774 dblp:conf/issta/FraserSMAP13 fatcat:slpbaz6jybe7lpoazratosllki

Scented Since the Beginning: On the Diffuseness of Test Smells in Automatically Generated Test Code

Giovanni Grano, Fabio Palomba, Dario Di Nucci, Andrea De Lucia, Harald Gall
2019 Journal of Systems and Software  
We also discover that test size is correlated with the generation of smelly tests.  ...  We also discover that test size is correlated with the generation of smelly tests.  ...  Test suite size is often correlated to the presence of Eager Test and Assertion Roulette.  ... 
doi:10.1016/j.jss.2019.07.016 fatcat:2q6irve72jdj3fwxt56hj44jm4

Code coverage and test suite effectiveness: Empirical study with real bugs in large systems

Pavneet Singh Kochhar, Ferdian Thung, David Lo
2015 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER)  
We use Randoop, a random test generation tool to generate test suites with varying levels of coverage and run them to analyse if the test suites can kill each of the real bugs or not.  ...  Our experiment finds that there is indeed statistically significant correlation between code coverage and bug kill effectiveness.  ...  Our study finds that test suite effectiveness is weakly to strongly correlated with test suite size, and test suite effectiveness is moderately to strongly correlated with coverage.  ... 
doi:10.1109/saner.2015.7081877 dblp:conf/wcre/KochharTL15 fatcat:lrma4wzb3vc6vnf53nbg3x5xni

Achieving scalable mutation-based generation of whole test suites

Gordon Fraser, Andrea Arcuri
2014 Empirical Software Engineering  
, and second we use whole test suite generation to optimize test suites towards killing the highest number of mutants, rather than selecting individual mutants.  ...  However, mutation based test generation is hampered because usually there are simply too many mutants, and too many of these are either trivially killed or equivalent.  ...  Acknowledgements This project has been funded by a Google Focused Research Award on "Test Amplification" and the Norwegian Research Council. References Acree AT (1980) On mutation.  ... 
doi:10.1007/s10664-013-9299-z fatcat:zjszitxffzfrtp24jy443a4hb4

A comparative study of programmer-written and automatically inferred contracts

Nadia Polikarpova, Ilinca Ciupa, Bertrand Meyer
2009 Proceedings of the eighteenth international symposium on Software testing and analysis - ISSTA '09  
Around a third of the generated assertions clauses are either incorrect or irrelevant.  ...  clauses) with the already present contracts.  ...  Some studies investigate the effect of the test suite on Daikoninferred contracts.  ... 
doi:10.1145/1572272.1572284 dblp:conf/issta/PolikarpovaCM09 fatcat:lcxbyf5b5rcejfh7pzbb3tbtsm

State Coverage: An Empirical Analysis Based on a User Study [chapter]

Dries Vanoverberghe, Emma Eyckmans, Frank Piessens
2013 Lecture Notes in Computer Science  
This paper presents a user-based experiment to evaluate whether the state coverage of a test suite correlates with the number defects it discovers.  ...  While most existing test adequacy criteria measure the degree of exploration of the code under test, state coverage estimates the strength of the assertions in the test suite.  ...  The advantage is that the new definition will more strongly comply with the expectations of the users.  ... 
doi:10.1007/978-3-642-35843-2_40 fatcat:b6rsfgqy6bf47kpd3ml4ojiygq

Using In-Process Testing Metrics to Estimate Post-Release Field Quality

Nachiappan Nagappan, Laurie Williams, Mladen Vouk, Jason Osborne
2007 The 18th IEEE International Symposium on Software Reliability (ISSRE '07)  
The metric suite is applicable for software products implemented in Java for which an extensive suite of automated unit test cases are incrementally created as development proceeds.  ...  This paper presents a suite of nine static in-process unit test metrics, the Software Testing and Reliability Early Warning for Java (STREW-J) metric suite.  ...  The number of asserts [31] and other metrics specific to xUnit-like test automation are used in the STREW-J metric suite.  ... 
doi:10.1109/issre.2007.18 dblp:conf/issre/NagappanWVO07 fatcat:7wpyr7tz4fdwnikazntt36pvgm

Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study

Gordon Fraser, Matt Staats, Phil McMinn, Andrea Arcuri, Frank Padberg
2015 ACM Transactions on Software Engineering and Methodology  
In the absence of a specification, developers are expected to manually construct or verify the test oracle for each test input.  ...  To investigate this issue, we performed two controlled experiments comparing a total of 97 subjects split between writing tests manually and writing tests with the aid of an automated unit test generation  ...  Thus as EVOSUITE derived test suites evolve to become more effective, extraneous tests/assertions are dropped and replaced with tests using a smaller number of assertions.  ... 
doi:10.1145/2699688 fatcat:4xadjanfdjbr5p63tgoez6zmhu

Early estimation of software quality using in-process testing metrics

Nachiappan Nagappan, Laurie Williams, Mladen Vouk, Jason Osborne
2005 Proceedings of the third workshop on Software quality - 3-WoSQ  
This paper presents the use of a suite of in-process metrics that leverages the software testing effort to provide (1) an estimation of potential software field quality in early software development phases  ...  A controlled case study conducted at North Carolina State University provides initial indication that our approach is effective for making an early assessment of post-release field quality.  ...  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.  ... 
doi:10.1145/1083292.1083304 dblp:conf/icse/NagappanWVO05 fatcat:2j42r2utsfhold47bx5f6u56fq

Early estimation of software quality using in-process testing metrics

Nachiappan Nagappan, Laurie Williams, Mladen Vouk, Jason Osborne
2005 Software engineering notes  
This paper presents the use of a suite of in-process metrics that leverages the software testing effort to provide (1) an estimation of potential software field quality in early software development phases  ...  A controlled case study conducted at North Carolina State University provides initial indication that our approach is effective for making an early assessment of post-release field quality.  ...  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.  ... 
doi:10.1145/1082983.1083304 fatcat:b7lxvfxz3zc2loghe33vrah34y

Selfless assertions: some empirical evidence

John Turri
2014 Synthese  
Selfless assertions are said to be intuitively compelling examples where agents should assert propositions that they don't even believe and, hence, don't know.  ...  This paper reports four experiments demonstrating that "selfless assertors" are viewed as both believing and knowing the propositions they assert: this is the natural and intuitive way of interpreting  ...  Critics have offered other examples of selfless assertion. But they are ill suited to test intuitions about assertability.  ... 
doi:10.1007/s11229-014-0621-0 fatcat:zoctbznbfbehjhlzazpewd6cji

PyNose: A Test Smell Detector For Python [article]

Tongjie Wang, Yaroslav Golubev, Oleg Smirnov, Jiawei Li, Timofey Bryksin, Iftekhar Ahmed
2021 arXiv   pre-print
Test smells have a detrimental effect not only on test code but also on the production code that is being tested.  ...  Similarly to production code, code smells also occur in test code, where they are called test smells.  ...  Other strongly connected pairs are all associated with Assertion Roulette due to its popularity. If a test suite has a Duplicate Assert, it has an Assertion Roulette in 93.1% of the cases.  ... 
arXiv:2108.04639v1 fatcat:ofysdqz5w5fl7eyghtipq5u67m

Improving test suites via operational abstraction

M. Harder, J. Mellen, M.D. Ernst
2003 25th International Conference on Software Engineering, 2003. Proceedings.  
The resulting test suites are as small, and detect as many faults, as suites with 100% branch coverage, and are better at detecting certain common faults.  ...  The technique dynamically generates operational abstractions (which describe observed behavior and are syntactically identical to formal specifications) from test suite executions.  ...  Acknowledgments Benjamin Morse wrote the C/C++ front end to the Daikon system and assisted with data collection.  ... 
doi:10.1109/icse.2003.1201188 dblp:conf/icse/HarderME03 fatcat:i2mmx7vp5vfotca35iu4z5zjkm
« Previous Showing results 1 — 15 out of 59,292 results