Filters








44 Hits in 5.3 sec

Statistically Model Checking PCTL Specifications on Markov Decision Processes via Reinforcement Learning [article]

Yu Wang, Nima Roohi, Matthew West, Mahesh Viswanathan, Geir E. Dullerud
2020 arXiv   pre-print
In this work, we focus on model checking PCTL specifications statistically on Markov Decision Processes (MDPs) by sampling, e.g., checking whether there exists a feasible policy such that the probability  ...  We use reinforcement learning to search for such a feasible policy for PCTL specifications, and then develop a statistical model checking (SMC) method with provable guarantees on its error.  ...  The statistical model checking of PCTL specifications on Markov Decision Processes (MDPs) is frequently encountered in many decision problems -e.g., for a robot in a grid world under probabilistic disturbance  ... 
arXiv:2004.00273v2 fatcat:xluhcbujb5dtjpfxedmrwmw6ju

Dependability Analysis of Deep Reinforcement Learning based Robotics and Autonomous Systems through Probabilistic Model Checking [article]

Yi Dong, Xingyu Zhao, Xiaowei Huang
2022 arXiv   pre-print
We then conduct Probabilistic Model Checking (PMC) on the designed DTMC to verify those properties.  ...  While Deep Reinforcement Learning (DRL) provides transformational capabilities to the control of Robotics and Autonomous Systems (RAS), the black-box nature of DRL and uncertain deployment environments  ...  ., Linear Temporal Logic (LTL) or PCTL. Then, via model checkers, a systematic exploration and analysis are performed to check if a claimed property holds.  ... 
arXiv:2109.06523v3 fatcat:tfkizgpwjbfefmtuerjoam75ge

Lifted Model Checking for Relational MDPs [article]

Wen-Chi Yang, Jean-François Raskin, Luc De Raedt
2022 arXiv   pre-print
It extends REBEL, a relational model-based reinforcement learning technique, toward relational pCTL model checking.  ...  Various frameworks handle relational domains, for instance, STRIPS planning and relational Markov Decision Processes.  ...  To this aim, relational Markov Decision Processes have been integrated with model checking principles for pCTL.  ... 
arXiv:2106.11735v2 fatcat:rhz4q7otqjao3o7uwiitwkyq4m

Deep Statistical Model Checking [chapter]

Timo P. Gros, Holger Hermanns, Jörg Hoffmann, Michaela Klauck, Marcel Steinmetz
2020 Lecture Notes in Computer Science  
Due to the possibility to model random noise in the decision actuation, each model instance induces a Markov decision process (MDP) as verification object.  ...  From the verification perspective, the externally learnt NN serves as a determinizer of the MDP, the result being a Markov chain which as such is amenable to statistical model checking.  ...  Background Markov Decision Processes. The models we consider are discrete-state Markov Decision Processes (MDP). For any nonempty set S we let D(S) denote the set of probability distribution over S.  ... 
doi:10.1007/978-3-030-50086-3_6 fatcat:hqnjedbyendnbkmogkituzdjim

Synthesizing Safe Policies under Probabilistic Constraints with Reinforcement Learning and Bayesian Model Checking [article]

Lenz Belzner, Martin Wirsing
2021 arXiv   pre-print
We show that an agent's confidence in constraint satisfaction provides a useful signal for balancing optimization and safety in the learning process.  ...  We introduce a framework for specification of requirements for reinforcement learners in constrained settings, including confidence about results.  ...  While our work builds on these ideas, policy synthesis is not a core aspect of statistical model checking: Usually information about the verification process is not induced into a learning process (28  ... 
arXiv:2005.03898v2 fatcat:kagyjmmzknfmzovbilzn37ufvm

Evolutionary-Guided Synthesis of Verified Pareto-Optimal MDP Policies

Simos Gerasimou, Javier Camara, Radu Calinescu, Naif Alasmari, Faisal Alhwikem, Xinwei Fang
2021 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)  
We present a new approach for synthesising Paretooptimal Markov decision process (MDP) policies that satisfy complex combinations of quality-of-service (QoS) software requirements.  ...  These policies correspond to optimal designs or configurations of software systems, and are obtained by translating MDP models of these systems into parametric Markov chains, and using multi-objective  ...  Markov Decision Processes Markov decision processes generalise DTMCs with the ability to model nondeterminism. Definition 2 (Markov decision process).  ... 
doi:10.1109/ase51524.2021.9678727 fatcat:kquqi7ahyrhtjaj3ux6qmmrjvi

The 10,000 Facets of MDP Model Checking [chapter]

Christel Baier, Holger Hermanns, Joost-Pieter Katoen
2019 Lecture Notes in Computer Science  
This paper presents a retrospective view on probabilistic model checking. We focus on Markov decision processes (MDPs, for short).  ...  We survey the basic ingredients of MDP model checking and discuss its enormous developments since the seminal works by Courcoubetis and Yannakakis in the early 1990s.  ...  Introduction Markov decision processes (MDPs) have their roots in operations research and stochastic control theory.  ... 
doi:10.1007/978-3-319-91908-9_21 fatcat:yjsuwb5ibjff3cq3niatu6sbxq

Probabilistic Model Checking of Robots Deployed in Extreme Environments [article]

Xingyu Zhao, Valentin Robu, David Flynn, Fateme Dinmohammadi, Michael Fisher, Matt Webster
2019 arXiv   pre-print
In this paper, we develop a framework for probabilistic model checking on a layered Markov model to verify the safety and reliability requirements of such robots, both at pre-mission stage and during runtime  ...  Two novel estimators based on conservative Bayesian inference and imprecise probability model with sets of priors are introduced to learn the unknown transition parameters from operational data.  ...  We thank Lorenzo Strigini, Peter Bishop and Andrey Povyakalo from City, University of London who provided insights on the initial ideas of the work.  ... 
arXiv:1812.04128v3 fatcat:iqw54ooxprcl3mf2x2hfrzsniq

Probabilistic Model Checking of Robots Deployed in Extreme Environments

Xingyu Zhao, Valentin Robu, David Flynn, Fateme Dinmohammadi, Michael Fisher, Matt Webster
2019 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
In this paper, we develop a framework for probabilistic model checking on a layered Markov model to verify the safety and reliability requirements of such robots, both at pre-mission stage and during runtime  ...  Two novel estimators based on conservative Bayesian inference and imprecise probability model with sets of priors are introduced to learn the unknown transition parameters from operational data.  ...  We thank Lorenzo Strigini, Peter Bishop and Andrey Povyakalo from City, University of London who provided insights on the initial ideas of the work.  ... 
doi:10.1609/aaai.v33i01.33018066 fatcat:isoakv3b6rgsvpfksyst2ojhdq

Control Improvisation with Probabilistic Temporal Specifications

Ilge Akkaya, Daniel J. Fremont, Rafael Valle, Alexandre Donze, Edward A. Lee, Sanjit A. Seshia
2016 2016 IEEE First International Conference on Internet-of-Things Design and Implementation (IoTDI)  
We learn from existing data a generative model (for instance, an explicit-duration hidden Markov model, or EDHMM) and then supervise this model in order to guarantee that the generated sequences satisfy  ...  some desirable specifications given in Probabilistic Computation Tree Logic (PCTL).  ...  Probabilistic Model Checking Our approach relies on the use of a verification method known as probabilistic model checking, which determines if a probabilistic model (such as a Markov chain or Markov decision  ... 
doi:10.1109/iotdi.2015.33 dblp:conf/iotdi/AkkayaFVDLS16 fatcat:xkajtd2b5vhevielprk3cwnhpi

Automated Verification and Synthesis of Stochastic Hybrid Systems: A Survey [article]

Abolfazl Lavaei, Sadegh Soudjani, Alessandro Abate, Majid Zamani
2022 arXiv   pre-print
In this survey, we overview the most recent results in the literature and discuss different approaches, including (in)finite abstractions, verification and synthesis for temporal logic specifications,  ...  stochastic similarity relations, (control) barrier certificates, compositional techniques, and a selection of results on continuous-time stochastic systems; we finally survey recently developed software  ...  survey article provides an introduction to the foundations of SHS, towards an easier understanding of many challenges and existing solutions related to formal verification and control synthesis of these models  ... 
arXiv:2101.07491v2 fatcat:dpir554ebfclhpj5m7e7fi2hv4

A storm is Coming: A Modern Probabilistic Model Checker [article]

Christian Dehnert and Sebastian Junges and Joost-Pieter Katoen and Matthias Volk
2017 arXiv   pre-print
We launch the new probabilistic model checker storm. It features the analysis of discrete- and continuous-time variants of both Markov chains and MDPs.  ...  Experiments on a variety of benchmarks show its competitive performance.  ...  Storm supports Markov chains and Markov decision processes (MDPs), both in two forms: discrete time and continuous time.  ... 
arXiv:1702.04311v1 fatcat:5oazkten7zg4zghea7wqf3s27q

Saturated Path-Constrained MDP: Planning under Uncertainty and Deterministic Model-Checking Constraints

Jonathan Sprauel, Andrey Kolobov, Florent Teichteil-Königsbuch
2014 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
This paper presents Saturated Path-Constrained Markov Decision Processes (SPC MDPs), a new MDP type for planning under uncertainty with deterministic model-checking constraints, e.g., "state s must be  ...  In this context, we propose a new problem class, Saturated Path-Constrained Markov Decision Processes (SPC MDPs), specifically designed to marry decision-theoretic planning to model-checking.  ...  Introduction Markov Decision Processes (MDPs) are some of the most popular models for optimizing the behavior of stochastic discrete-time dynamical systems.  ... 
doi:10.1609/aaai.v28i1.9041 fatcat:74vbkkvg55c3tel5s7ku4wbvjm

Survey on learning-based formal methods: Taxonomy, Applications and Possible future directions

Fujun Wang, Zining Cao, Lixing Tan, Hui Zong
2020 IEEE Access  
INDEX TERMS Formal methods, formal specification, formal verification, learning model, learning specification. 108562 VOLUME 8, 2020  ...  in learning models and/or specifications from observed system behaviors automatically.  ...  (DTMC) [18] , Markov decision process (MDP) [18] will not be introduced here due to space limitations, we refer readers to related references for details.  ... 
doi:10.1109/access.2020.3000907 fatcat:uiy7d2ellrc4jmzn4eumleeoy4

An investigation into serotonergic and environmental interventions against depression in a simulated delayed reward paradigm [article]

Bernd Porr, Alex Trew, Alice Miller
2019 bioRxiv   pre-print
We show with both standard behavioural simulations and model checking that SSRIs perform significantly better against interventions with psychedelics.  ...  for smaller signals but amplifying larger ones.  ...  This is achieved in Prism via the use of action labels. Specifically all synchronised transitions have the action label ([timed]).  ... 
doi:10.1101/580456 fatcat:atqlz66h2fblllj6dfcef4t6bu
« Previous Showing results 1 — 15 out of 44 results