Automatically testing string solvers

Alexandra Bugariu, Peter Müller
<span title="2020-06-27">2020</span> <i title="ACM"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/wvv27s77dvd5flktsj246kcxwu" style="color: black;">Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering</a> </i> &nbsp;
SMT solvers are at the basis of many applications, such as program verification, program synthesis, and test case generation. For all these applications to provide reliable results, SMT solvers must answer queries correctly. However, since they are complex, highlyoptimized software systems, ensuring their correctness is challenging. In particular, state-of-the-art testing techniques do not reliably detect when an SMT solver is unsound. In this paper, we present an automatic approach for
more &raquo; ... ng test cases that reveal soundness errors in the implementations of string solvers, as well as potential completeness and performance issues. We synthesize input formulas that are satisfiable or unsatisfiable by construction and use this ground truth as test oracle. We automatically apply satisfiability-preserving transformations to generate increasingly-complex formulas, which allows us to detect many errors with simple inputs and, thus, facilitates debugging. The experimental evaluation shows that our technique effectively reveals bugs in the implementation of widely-used SMT solvers and applies also to other types of solvers, such as automatabased solvers. We focus on strings here, but our approach carries over to other theories and their combinations. CCS CONCEPTS • Software and its engineering → Software testing and debugging.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/3377811.3380398">doi:10.1145/3377811.3380398</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/icse/Bugariu020.html">dblp:conf/icse/Bugariu020</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/yyiirnlegvgttpcpu4sdg4bppa">fatcat:yyiirnlegvgttpcpu4sdg4bppa</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20201107094658/https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/397450/paper.pdf;jsessionid=17019A30CF40F267F25041D70DCB4D27?sequence=1" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/58/5d/585df303a419e2f7812fdfa74d8354a515b8bdb0.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/3377811.3380398"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> acm.org </button> </a>