### Synthesis for PCTL in Parametric Markov Decision Processes [chapter]

Ernst Moritz Hahn, Tingting Han, Lijun Zhang
2011 Lecture Notes in Computer Science
In parametric Markov Decision Processes (PMDPs), transition probabilities are not fixed, but are given as functions over a set of parameters. A PMDP denotes a family of concrete MDPs. This paper studies the synthesis problem for PCTL in PMDPs: Given a specification Φ in PCTL, we synthesise the parameter valuations under which Φ is true. First, we divide the possible parameter space into hyper-rectangles. We use existing decision procedures to check whether Φ holds on each of the Markov
more » ... represented by the hyper-rectangle. As it is normally impossible to cover the whole parameter space by hyper-rectangles, we allow a limited area to remain undecided. We also consider an extension of PCTL with reachability rewards. To demonstrate the applicability of the approach, we apply our technique on a case study, using a preliminary implementation. Introduction Markov processes [6, 26] have been applied successfully to reason about quantitative properties in networked, distributed, and recently biological systems. This paper considers parametric Markov processes  , in which transition probabilities are not fixed, but depend on a set of parameters. As an example, consider a communication network with a lossy channel, where whenever a package is sent, it is received with probability x but lost with probability 1 − x. In this context, we are interested in, for instance, determining the parametric reachability probability with respect to a given set of states. This probability is a function in x. By inserting an appropriate value for x in the function, we will obtain a concrete model without parameters. The synthesis problem asks, for example, what are the possible parameter valuations such that the reachability probability is below the a priori specified threshold. Daws has devised a language-theoretic approach to solve the reachability problem in parametric Markov chains  . In this approach, the transition probabilities are considered as letters of an alphabet. Thus, the model is viewed as a finite automaton. Based on the state elimination approach , the regular expression describing the language of such an automaton is computed. In a postprocessing step, this regular expression is transformed into a rational function over the parameters of the model. In previous works , we have improved this