Symstra: A Framework for Generating Object-Oriented Unit Tests Using Symbolic Execution
Lecture Notes in Computer Science
Object-oriented unit tests consist of sequences of method invocations. Behavior of an invocation depends on the method's arguments and the state of the receiver at the beginning of the invocation. Correspondingly, generating unit tests involves two tasks: generating method sequences that build relevant receiverobject states and generating relevant method arguments. This paper proposes Symstra, a framework that achieves both test generation tasks using symbolic execution of method sequences with
... symbolic arguments. The paper defines symbolic states of object-oriented programs and novel comparisons of states. Given a set of methods from the class under test and a bound on the length of sequences, Symstra systematically explores the object-state space of the class and prunes this exploration based on the state comparisons. Experimental results show that Symstra generates unit tests that achieve higher branch coverage faster than the existing test-generation techniques based on concrete method arguments. Background Generating test sequences involves two tasks: generating method sequences that build relevant receiver-object state and generating relevant method arguments. Researchers have addressed this problem several times. Most tools generate test sequences using concrete representations. A popular approach is to use (smart) random generation; this approach is embodied in tools such as Jtest  (a commercial tool for Java) or JCrasher  and Eclat  (two research prototypes for Java). Random tests generated by these tools often execute the same sequences  and are not covering (do not cover all sequences). The AsmLT model-based testing tool [15, 16] uses concrete-state space-exploration techniques  to generate covering method sequences. But AsmLT requires the user to carefully choose sufficiently large concrete domains for method arguments and the right abstraction functions to guarantee the covering. Tools such as Korat  are able to generate non-isomorphic object graphs that can be used for testing, but they do not generate covering test sequences. King proposed in the 70's to use symbolic execution for testing and verification  . Because of the advances in constraint solvers, this technique recently regained the attention for test generation. For example, the BZ-TT tool uses constraint solving to derive method sequences from B specifications  . However, the B specifications are not object-oriented. Khurshid et al. [19, 33] proposed an approach for generating tests for Java classes based on symbolic execution. They show that their generation based on symbolic execution generates tests faster than their model checking of method sequences with concrete arguments. This is expected: symbolic representations describe not only single states, but sets of states, and when applicable, symbolic representations can yield large improvements, witnessed for example by symbolic model checking  . The approach of Khurshid et al. [19, 33] , however, generates the receiver-object states, similar to Korat , only as object graphs, not through method sequences. Moreover, it requires the user to provide specially constructed class invariants  , which effectively describe an over-approximation of the set of reachable object graphs. Symbolic execution is the foundation of static code analysis tools. These tools typically do not generate test data, but automatically verify simple properties of programs. These properties often allow merging symbolic states that stem from different execution paths. However, for test generation, states have to be kept separate, since different tests should be used for different paths. Recently, tools such as SLAM [2, 4] and Blast [7, 17] were adapted for test generation. However, neither of them can deal with complex data structures, which are the focus of this paper. Contributions This paper makes the following contributions. Symbolic Sequence Exploration: We propose Symstra, a framework that uses symbolic execution to generate method sequences. When applicable, Symstra uses an exhaustive exploration of method sequences with symbolic variables for primitive-type arguments. (We also discuss how Symstra can handle reference-type arguments.) Each symbolic argument represents a set of all possible concrete values for the argument.