An empirical framework for comparing effectiveness of testing and property-based formal analysis

Jeremy S. Bradbury, James R. Cordy, Juergen Dingel
<span title="">2005</span> <i title="ACM Press"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/5a67ttu6pvdo7msdeau5nqnm5q" style="color: black;">The 6th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering - PASTE &#39;05</a> </i> &nbsp;
Today, many formal analysis tools are not only used to provide certainty but are also used to debug software systems -a role that has traditional been reserved for testing tools. We are interested in exploring the complementary relationship as well as tradeoffs between testing and formal analysis with respect to debugging and more specifically bug detection. In this paper we present an approach to the assessment of testing and formal analysis tools using metrics to measure the quantity and
more &raquo; ... iency of each technique at finding bugs. We also present an assessment framework that has been constructed to allow for symmetrical comparison and evaluation of tests versus properties. We are currently beginning to conduct experiments and this paper presents a discussion of possible outcomes of our proposed empirical study.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/1108792.1108795">doi:10.1145/1108792.1108795</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/paste/BradburyCD05.html">dblp:conf/paste/BradburyCD05</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/hu5devxj4jh4lnv3coxccuvcba">fatcat:hu5devxj4jh4lnv3coxccuvcba</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170809022031/http://research.cs.queensu.ca/home/cordy/Papers/BCD_PASTE05_EmpFrame.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/21/17/21173e873a2490702601185c8dd24bb92733cb13.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/1108792.1108795"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> acm.org </button> </a>