Measuring a Java Test Suite Coverage Using JML Specifications

F. Dadeau, Y. Ledru, L. du Bousquet
2007 Electronical Notes in Theoretical Computer Science  
We propose in this paper a way to measure the coverage of a Java test suite by considering the JML specification associed to the Java program under test. This approach is based on extracting a predicatebased graph from the JML method specifications. We then measure the coverage of this latter w.r.t. nodes of the graph that are visited by the test suite. In addition, we propose to check whether the test suite satisfies classical condition coverage criteria. We also introduce a tool, to be used
more » ... precompiler for Java, that is in charge of measuring and reporting the coverage according to these criteria. Open access under CC BY-NC-ND license. The recent arise of annotation languages makes it possible to specify the behavior of programs (i.e. of methods) inside the source code in terms of pre-and postconditions. It provides another "vision" of what a method should do, which can also be seen as expressing low-level requirements. It also provides a black-box view of what a method should do, expressed in terms of a contract [20] . Model-based testing [2] consists in computing test suites from a model of the considered program or system. Model-based conformance testing consists in ensuring that the program does not have an unintended behavior w.r.t. its specification. This conformance can be observed through observation points or using a run-time assertion checking mechanism if the proximity of both the specification and the program makes it possible. In this context, the Java Modeling Language [16] (JML) has been introduced to act as a behavioral interface specification language (BISL) for Java programs. JML can be used as an oracle for testing, considering that if no JML assertion is ever violated during the program execution, then the test succeeds, otherwise, it fails. A previous work [3] has introduced the principles of model-based testing from JML specifications. Thus, some JML-based coverage criteria were used to guide the test target definition. We came to the idea that the coverage criteria defined in the latter work could be used to evaluate test suites that would have been produced by several tools, using different approaches, such as combinatorial testing (e.g. JMLUnit [6], Tobias [18]) or random testing (e.g. Jartege [23]). We propose an approach for evaluating test suites for Java programs w.r.t. the coverage of an associated JML specification, expressing the behavior and/or the requirements of the methods. In addition, we propose to check different condition coverage criteria, that are contained within the disjunctions of the predicates of the specification. The paper is organized as follows. Section 2 introduces the modeling possibilities provided by the Java Modeling Language. The first coverage criterion, based on the method specifications, is presented in Sect. 3. Section 4 is dedicated to the condition coverage definition. The principles of the measure and especially the implementation and the experiments are detailed in Sect. 5. Section 6 presents the related work, before concluding and providing a glimpse of the future work in Sect. 7.
doi:10.1016/j.entcs.2007.08.003 fatcat:2jyofr6k7felxada4k4bzmdzo4