Formal mutation testing for Circus

Alex Alberto, Ana Cavalcanti, Marie-Claude Gaudel, Adenilso Simão
<span title="">2017</span> <i title="Elsevier BV"> <a target="_blank" rel="noopener" href="" style="color: black;">Information and Software Technology</a> </i> &nbsp;
Context: The demand from industry for more dependable and scalable test-development mechanisms has fostered the use of formal models to guide the generation of tests. Despite many advancements having been obtained with state-based models, such as Finite State Machines (FSMs) and Input/Output Transition Systems (IOTSs), more advanced formalisms are required to specify large, state-rich, concurrent systems. Circus, a state-rich process algebra combining Z, CSP and a refinement calculus, is
more &raquo; ... e for this; however, deriving tests from such models is accordingly more challenging. Recently, a testing theory has been stated for Circus, allowing the verification of process refinement based on exhaustive test sets. Objective: We investigate fault-based testing for refinement from Circus specifications using mutation. We seek the benefits of such techniques in test-set quality assertion and fault-based test-case selection. We target results relevant not only for Circus, but to any process algebra for refinement that combines CSP with a data language. Method: We present a formal definition for fault-based test sets, extending the Circus testing theory, and an extensive study of mutation operators for Circus. Using these results, we propose an approach to generate tests to kill mutants. Finally, we explain how prototype tool support can be obtained with the implementation of a mutant generator, a translator from Circus to CSP, and a refinement checker for CSP, and with a more sophisticated chain of tools that support the use of symbolic tests. Results: We formally characterise mutation testing for Circus, defining the exhaustive test sets that can kill a given mutant. We also provide a technique to select tests from these sets based on specification traces of the mutants. Finally, we present mutation operators that consider faults related to both reactive and data manipulation behaviour. Altogether, we define a new fault-based test-generation technique for Circus. Conclusion: We conclude that mutation testing for Circus can truly aid making test generation from state-rich model more tractable, by focussing on particular faults. 45 considering the specific features and particularities of Circus. Moreover, our results are valid in the context of other process algebras, especially those based on CSP [34, 35] . The contribution of this paper is manifold. First, we instantiate the notions of mutation testing for a state-rich concurrent language, namely, Circus, and its formal 2 theory of testing. In particular, we face the challenge of associating mutations in the 50 text of a Circus specification to traces of the Circus denotational semantics that define tests that cover the mutation. Even though mutation testing has already been applied to languages and theories upon which Circus is based, such as CSP [31] and the UTP [36], the consideration of a state-rich process algebra for refinement with a UTP semantics is novel. Second, we propose mutation operators for Circus, analysing and adapting 55 existing ones for the underlying languages and designing some that are specific to Circus. Third, we describe prototype tool support for the application of the mutant operators and two approaches to generate tests that can kill these mutants. When it is feasible to translate the considered Circus specification into CSP, we propose the use of the FDR model checker. For the other cases, we identify a tool chain that copes 60 directly with Circus specifications via slicing techniques and symbolic execution. This paper is organized as follows. Section 2 gives an overview of the aspects of Circus and its testing theory that we use here. Section 3 extends the testing theory to consider mutation testing and describes our approach to generating tests based on mutants. The mutation operators used to generate the mutants themselves are defined 65 in Section 4. Tool support for automation of our approach is discussed in Section 5, and an extra complete example is introduced in Section 6. Finally, we present some related and future work and conclusions in Sections 7 and 8. Circus and its testing theory In this section, we give a brief description of the Circus language, its operational 70 semantics [26] , and its testing theory [22] . Circus notation and operational semantics As exemplified is Figure 1 , Circus allows us to model systems and their components via (a network of) interacting processes. In Figure 1 , we define a single process Chrono that specifies the reactive behaviour of a chronometer. This is a process that 75
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="">doi:10.1016/j.infsof.2016.04.003</a> <a target="_blank" rel="external noopener" href="">fatcat:5437igboo5fophxarvu7clhqda</a> </span>
<a target="_blank" rel="noopener" href="" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href=""> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> </button> </a>