Automatic testing of symbolic execution engines via program generation and differential testing

Timotej Kapus, Cristian Cadar
2017 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE)  
Symbolic execution has attracted significant attention in recent years, with applications in software testing, security, networking and more. Symbolic execution tools, like CREST, KLEE, FuzzBALL, and Symbolic PathFinder, have enabled researchers and practitioners to experiment with new ideas, scale the technique to larger applications and apply it to new application domains. Therefore, the correctness of these tools is of critical importance. In this paper, we present our experience extending
more » ... mpiler testing techniques to find errors in both the concrete and symbolic execution components of symbolic execution engines. The approach used relies on a novel way to create program versions, in three different testing modes-concrete, single-path and multi-path-each exercising different features of symbolic execution engines. When combined with existing program generation techniques and appropriate oracles, this approach enables differential testing within a single symbolic execution engine. We have applied our approach to the KLEE, CREST and FuzzBALL symbolic execution engines, where it has discovered 20 different bugs exposing a variety of important errors having to do with the handling of structures, division, modulo, casting, vector instructions and more, as well as issues related to constraint solving, compiler optimisations and test input replay.
doi:10.1109/ase.2017.8115669 dblp:conf/kbse/KapusC17 fatcat:xq7suq3zqnbytiztwd3w7jozlm