Energy and Mean-Payoff Parity Markov Decision Processes [chapter]

Krishnendu Chatterjee, Laurent Doyen
2011 Lecture Notes in Computer Science  
We consider Markov Decision Processes (MDPs) with mean-payoff parity and energy parity objectives. In system design, the parity objective is used to encode ω-regular specifications, while the mean-payoff and energy objectives can be used to model quantitative resource constraints. The energy condition requires that the resource level never drops below 0, and the mean-payoff condition requires that the limit-average value of the resource consumption is within a threshold. While these two (energy
more » ... and mean-payoff) classical conditions are equivalent for two-player games, we show that they differ for MDPs. We show that the problem of deciding whether a state is almost-sure winning (i.e., winning with probability 1) in energy parity MDPs is in NP ∩ coNP, while for meanpayoff parity MDPs, the problem is solvable in polynomial time.
doi:10.1007/978-3-642-22993-0_21 fatcat:vqqlf5mccndlvn4k2rlkms3ptq