Filters








116 Hits in 3.5 sec

Enforcing Almost-Sure Reachability in POMDPs [article]

Sebastian Junges, Nils Jansen, Sanjit A. Seshia
2021 arXiv   pre-print
We consider the EXPTIME-hard problem of synthesising policies that almost-surely reach some goal state without ever visiting a bad state.  ...  In particular, we are interested in computing the winning region, that is, the set of system configurations from which a policy exists that satisfies the reachability specification.  ...  We evaluate the shield by letting an agent explore the POMDP environment according to the permissive policy, thereby enforcing the satisfaction of the almost-sure specification.  ... 
arXiv:2007.00085v3 fatcat:kpx2ysnfzbduljymrde3tss6da

Enforcing Almost-Sure Reachability in POMDPs [chapter]

Sebastian Junges, Nils Jansen, Sanjit A. Seshia
2021 Lecture Notes in Computer Science  
We consider the EXPTIME-hard problem of synthesising policies that almost-surely reach some goal state without ever visiting a bad state.  ...  In particular, we are interested in computing the winning region, that is, the set of system configurations from which a policy exists that satisfies the reachability specification.  ...  POMDP Almost-Sure Reachability Verification.  ... 
doi:10.1007/978-3-030-81688-9_28 fatcat:g2iwyuuxsrcbxkli46wjopib6m

Enforcing Almost-Sure Reachability in POMDPs [article]

Sebastian Junges, Nils Jansen, Sanjit A. Seshia
2020
We consider the EXPTIME-hard problem of synthesising policies that almost-surely reach some goal state without ever visiting a bad state.  ...  In particular, we are interested in computing the winning region, that is, the set of system configurations from which a policy exists that satisfies the reachability specification.  ...  POMDP Almost-Sure Reachability Verification.  ... 
doi:10.48550/arxiv.2007.00085 fatcat:lfy4fr4ngvgdrfmifg5lhti56q

Sensor Synthesis for POMDPs with Reachability Objectives [article]

Krishnendu Chatterjee, Martin Chmelik, Ufuk Topcu
2017 arXiv   pre-print
that almost-surely (with probability~1) satisfies a reachability objective.  ...  Partially observable Markov decision processes (POMDPs) are widely used in probabilistic planning problems in which an agent interacts with an environment using noisy and imprecise sensors.  ...  the reachability objective almost-surely.  ... 
arXiv:1710.00675v1 fatcat:qli6oooiunfkdegu6fulifnrl4

A Symbolic SAT-based Algorithm for Almost-sure Reachability with Small Strategies in POMDPs [article]

Krishnendu Chatterjee and Martin Chmelik and Jessica Davies
2015 arXiv   pre-print
We study the problem of almost-sure reachability, where given a set of target states, the question is to decide whether there is a policy to ensure that the target set is reached with probability 1 (almost-surely  ...  POMDPs are standard models for probabilistic planning problems, where an agent interacts with an uncertain environment.  ...  This results in a practical, symbolic algorithm for the almost-sure reachability problem in POMDPs.  ... 
arXiv:1511.08456v1 fatcat:xafl4tcpqbgh7pemmqvqdxdcxy

What is decidable about partially observable Markov decision processes with ω -regular objectives

Krishnendu Chatterjee, Martin Chmelík, Mathieu Tracol
2016 Journal of computer and system sciences (Print)  
The most prominent and important open question is whether the almost-sure and positive winning problems are decidable for parity and Muller objectives in POMDPs under finite-memory strategies.  ...  undecidability for positive winning for Büchi and almost-sure winning for coBüchi objectives was established in [1, 2].  ...  By Lemma 22 there exists a finite-memory almost-sure winning strategy σ in the POMDP G. Let us consider the almost-sure winning projected strategy σ ′ = proj (σ) in the POMDP G.  ... 
doi:10.1016/j.jcss.2016.02.009 fatcat:ofxjveza2nelxal5yotjj2xbum

What is Decidable about Partially Observable Markov Decision Processes with ω-Regular Objectives [article]

Krishnendu Chatterjee, Martin Chmelik, Mathieu Tracol
2013 arXiv   pre-print
The class of ω-regular languages extends regular languages to infinite strings and provides a robust specification language to express all properties used in verification, and parity objectives are canonical  ...  We consider partially observable Markov decision processes (POMDPs) with ω-regular conditions specified as parity objectives.  ...  By Lemma 22 there exists a finite-memory almost-sure winning strategy σ in the POMDP G. Let us consider the almost-sure winning projected strategy σ ′ = proj (σ) in the POMDP G.  ... 
arXiv:1309.2802v1 fatcat:pu7i5ysrgje7toisabfkytvpbi

Stochastic Shortest Path with Energy Constraints in POMDPs [article]

Tomáš Brázdil, Krishnendu Chatterjee, Martin Chmelík, Anchit Gupta, Petr Novotný
2016 arXiv   pre-print
The energy levels may increase and decrease with transitions, and the hard constraint requires that the energy level must remain positive in all steps till the target is reached.  ...  First, we present a novel algorithm for solving POMDPs with energy levels, developing on existing POMDP solvers and using RTDP as its main method.  ...  In particular, for every policy σ ∈ EnSafe M T (E, cap) that almost surely reaches T one can construct a policyσ in M × such that σ almost surely reaches T × and Eσ[TC It follows that to solve the qualitative  ... 
arXiv:1602.07565v2 fatcat:phy325vc2vd55ebs7ittaa65qi

Active Diagnosis for Probabilistic Systems [chapter]

Nathalie Bertrand, Éric Fabre, Stefan Haar, Serge Haddad, Loïc Hélouët
2014 Lecture Notes in Computer Science  
The safe active diagnosis problem is similar, but aims at enforcing diagnosability while preserving a positive probability to non faulty runs, i.e. without enforcing the occurrence of a fault.  ...  This system is then diagnosable if the fault can always be detected, and the active diagnosis problem consists in controlling the system in order to ensure its diagnosability.  ...  The notion of a safely diagnosable pLTS is introduced to ensure that fault occurrence is not almost sure.  ... 
doi:10.1007/978-3-642-54830-7_2 fatcat:fhcixh3iejclbpfyjgnoqeatpq

Parameter Synthesis in Markov Models: A Gentle Survey [article]

Nils Jansen, Sebastian Junges, Joost-Pieter Katoen
2022 arXiv   pre-print
Acknowledgements We thank all our co-authors in the work(s) surveyed in this paper: Erika Ábrahám, Christel Baier, Bernd Becker, Harold Bruintjes, Florian Corzilius, Murat Cubuktepe, Christian Hensel,  ...  The second line encodes that s 3 reaches itself almost surely, and that s 3 can almost surely not be reached from s 0 .  ...  synthesis problem for ϕ in POMDPs.  ... 
arXiv:2207.06801v1 fatcat:jbhv52tpdbebflxvqcxjser564

Under-Approximating Expected Total Rewards in POMDPs

Alexander Nikolai Bork, Joost-Pieter Katoen, Tim Quatmann
2022 Tools and Algorithms for the Construction and Analysis of Systems : 28th International Conference  
We consider the problem: is the optimal expected total reward to reach a goal state in a partially observable Markov decision process (POMDP) below a given threshold?  ...  This is done by abstracting finite unfoldings of the infinite belief MDP of the POMDP. The key issue is to find a suitable under-approximation of the value function.  ...  Dedicated verification techniques for the qualitative setting-almost-sure reachability-are presented in [17, 16, 27] . Experimental results.  ... 
doi:10.18154/rwth-2022-03987 fatcat:g3rl7ksiu5btflinhdn6dgfcau

Probabilistic ω-automata

Christel Baier, Marcus Grösser, Nathalie Bertrand
2012 Journal of the ACM  
(2) the almost-sure semantics that requires acceptance with probability 1, and (3) the threshold semantics that relies on an additional parameter λ ∈]0, 1[ that specifies a lower probability bound for  ...  Vice-versa, the decididability of the emptiness problem for PBA =1 is a consequence of a more general result, the decidability of partially observable Markov decision processes under almost-sure Büchi  ...  Decidability Results Before we show any decidability results for POMDP, we first prove that almost-sure repeated reachability and almost-sure reachability are interreducible for POMDP.  ... 
doi:10.1145/2108242.2108243 fatcat:k4qob2zm4jh7rpz52eaziuzazu

Supervisor Synthesis of POMDP based on Automata Learning [article]

Xiaobin Zhang, Bo Wu, Hai Lin
2017 arXiv   pre-print
However, its comprehensiveness makes the planning and control in POMDP difficult.  ...  An example is given in detailed steps to illustrate the supervisor synthesis algorithm.  ...  In [29] , the authors use observation-stationary (memoryless) con-troller to regulate POMDP to satisfy almost-sure reachability properties.  ... 
arXiv:1703.08262v1 fatcat:x4kbup72ajd3ja4th4ioojk7ve

Task-Guided Inverse Reinforcement Learning Under Partial Information [article]

Franck Djeumou, Murat Cubuktepe, Craig Lennon, Ufuk Topcu
2021 arXiv   pre-print
., computing an optimal policy given a reward function, in POMDPs.  ...  We remove this assumption by developing an algorithm for IRL in partially observable Markov decision processes (POMDPs).  ...  Enforcing Yu, H.; and Bertsekas, D. P. 2008. On Near Optimality of the Almost-Sure Reachability in POMDPs. arXiv preprint. Set of Finite-State Controllers for Average Cost POMDP.  ... 
arXiv:2105.14073v2 fatcat:5kluysmvurcdvmxqupbscbifle

Safe Reinforcement Learning via Shielding for POMDPs [article]

Steven Carr, Nils Jansen, Sebastian Junges, Ufuk Topcu
2022 arXiv   pre-print
The standard models to capture scenarios with limited sensing are partially observable Markov decision processes (POMDPs). Safe RL for these models remains an open problem so far.  ...  Reinforcement learning (RL) in safety-critical environments requires an agent to avoid decisions with catastrophic consequences.  ...  Such specifications necessitate to always avoid certain bad states from AVOID ⊆ S and reach states from REACH ⊆ S almost-surely, i.e., with probability one (for arbitrary long horizons).  ... 
arXiv:2204.00755v1 fatcat:f4mb5lklijfqbmc526ilhxhuja
« Previous Showing results 1 — 15 out of 116 results