Percentile Queries in Multi-Dimensional Markov Decision Processes [article]

Mickael Randour and Jean-François Raskin and Ocan Sankur
2016 arXiv   pre-print
Markov decision processes (MDPs) with multi-dimensional weights are useful to analyze systems with multiple objectives that may be conflicting and require the analysis of trade-offs. We study the complexity of percentile queries in such MDPs and give algorithms to synthesize strategies that enforce such constraints. Given a multi-dimensional weighted MDP and a quantitative payoff function f, thresholds v_i (one per dimension), and probability thresholds α_i, we show how to compute a single
more » ... egy to enforce that for all dimensions i, the probability of outcomes ρ satisfying f_i(ρ) ≥ v_i is at least α_i. We consider classical quantitative payoffs from the literature (sup, inf, lim sup, lim inf, mean-payoff, truncated sum, discounted sum). Our work extends to the quantitative case the multi-objective model checking problem studied by Etessami et al. in unweighted MDPs.
arXiv:1410.4801v3 fatcat:pgwnk53kqzaxbapfhautnso7zm