Conformance test experiments for distributed real-time systems
Software engineering notes
This paper introduces a new technique for testing that a distributed real-time system satisfies a formal timed automata specification. It outlines how to write test specifications in the language of Uppaal timed automata, how to translate those specifications into program code for executing the tests, and describes the results of test experiments on a distributed real-time system with limited hardware and software resources. Section 2 outlines the test technique used in this paper, and Section
... describes a typical distributed real-time system which is used as running example for the paper. Section 4 explains how tests are specified in the formal language of Uppaal timed automata. Section 5 describes our implementation environment and Section 6 how to translate test specifications into test programs. Three test experiments are described in Section 7. TEST TECHNIQUE The test technique presented in this paper is based on finite state machine conformance test methods [4, 5, 9] . The desired behaviour of a system under test (SUT) is given by a formal Uppaal timed automata specification  . The tester generates test cases from the formal specification and executes them to determine whether the behaviour of the SUT matches this desired behaviour. Our main test purpose is to find defects in the SUT, and so test cases are chosen to exercise, as far as possible, parts of the system which could exhibit defects. Our test technique can also be used for reliability testing, in which the tester observes "normal" activity with the purpose of increasing confidence that the system reliably meets its specification, but in this paper our focus is on defect testing. A defect is defined as any SUT deviation in timing or value from the expected behaviour described by an Uppaal timed automata (UTA) specification. In our test method, the value and timing of all inputs relevant to a test purpose is defined explicitly and completely by its UTA specification. That is, test specifications are input closed. All outputs which constitute correct response to the inputs are also explicitly defined by the UTA test specification. An incorrect output, known as a fault or a defect, is any output which does not match the value and timing explicitly specified in the behaviour. That is, we define defects implicitly rather than explicitly as in  . EXAMPLE SYSTEM UNDER TEST Our test method is designed for distributed real-time systems consisting of a set of tasks communicating by shared variables and messages, distributed over two or more processors. The example system used in this paper, adapted from Braberman and Felder  , is typical of this class of systems. It consists of two processors which monitor inputs from several sources, analyse them and control a valve output which can be either opened or closed.