A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Statistically Model Checking PCTL Specifications on Markov Decision Processes via Reinforcement Learning
[article]
2020
arXiv
pre-print
Preserved Fulltext
Other Versions
In this work, we focus on model checking PCTL specifications statistically on Markov Decision Processes (MDPs) by sampling, e.g., checking whether there exists a feasible policy such that the probability ...
We use reinforcement learning to search for such a feasible policy for PCTL specifications, and then develop a statistical model checking (SMC) method with provable guarantees on its error. ...
The statistical model checking of PCTL specifications on Markov Decision Processes (MDPs) is frequently encountered in many decision problems -e.g., for a robot in a grid world under probabilistic disturbance ...
arXiv:2004.00273v2
fatcat:xluhcbujb5dtjpfxedmrwmw6ju
Dependability Analysis of Deep Reinforcement Learning based Robotics and Autonomous Systems through Probabilistic Model Checking
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We then conduct Probabilistic Model Checking (PMC) on the designed DTMC to verify those properties. ...
While Deep Reinforcement Learning (DRL) provides transformational capabilities to the control of Robotics and Autonomous Systems (RAS), the black-box nature of DRL and uncertain deployment environments ...
., Linear Temporal Logic (LTL) or PCTL. Then, via model checkers, a systematic exploration and analysis are performed to check if a claimed property holds. ...
arXiv:2109.06523v3
fatcat:tfkizgpwjbfefmtuerjoam75ge
Lifted Model Checking for Relational MDPs
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
It extends REBEL, a relational model-based reinforcement learning technique, toward relational pCTL model checking. ...
Various frameworks handle relational domains, for instance, STRIPS planning and relational Markov Decision Processes. ...
To this aim, relational Markov Decision Processes have been integrated with model checking principles for pCTL. ...
arXiv:2106.11735v2
fatcat:rhz4q7otqjao3o7uwiitwkyq4m
Deep Statistical Model Checking
[chapter]
2020
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Due to the possibility to model random noise in the decision actuation, each model instance induces a Markov decision process (MDP) as verification object. ...
From the verification perspective, the externally learnt NN serves as a determinizer of the MDP, the result being a Markov chain which as such is amenable to statistical model checking. ...
Background Markov Decision Processes. The models we consider are discrete-state Markov Decision Processes (MDP). For any nonempty set S we let D(S) denote the set of probability distribution over S. ...
doi:10.1007/978-3-030-50086-3_6
fatcat:hqnjedbyendnbkmogkituzdjim
Synthesizing Safe Policies under Probabilistic Constraints with Reinforcement Learning and Bayesian Model Checking
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2020 pre-print arXiv:2005.03898v1 fatcat:627eymbrxzbbjal7ag4drhgh4e
We show that an agent's confidence in constraint satisfaction provides a useful signal for balancing optimization and safety in the learning process. ...
We introduce a framework for specification of requirements for reinforcement learners in constrained settings, including confidence about results. ...
While our work builds on these ideas, policy synthesis is not a core aspect of statistical model checking: Usually information about the verification process is not induced into a learning process (28 ...
arXiv:2005.03898v2
fatcat:kagyjmmzknfmzovbilzn37ufvm
Evolutionary-Guided Synthesis of Verified Pareto-Optimal MDP Policies
2021
2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
We present a new approach for synthesising Paretooptimal Markov decision process (MDP) policies that satisfy complex combinations of quality-of-service (QoS) software requirements. ...
These policies correspond to optimal designs or configurations of software systems, and are obtained by translating MDP models of these systems into parametric Markov chains, and using multi-objective ...
Markov Decision Processes Markov decision processes generalise DTMCs with the ability to model nondeterminism.
Definition 2 (Markov decision process). ...
doi:10.1109/ase51524.2021.9678727
fatcat:kquqi7ahyrhtjaj3ux6qmmrjvi
The 10,000 Facets of MDP Model Checking
[chapter]
2019
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This paper presents a retrospective view on probabilistic model checking. We focus on Markov decision processes (MDPs, for short). ...
We survey the basic ingredients of MDP model checking and discuss its enormous developments since the seminal works by Courcoubetis and Yannakakis in the early 1990s. ...
Introduction Markov decision processes (MDPs) have their roots in operations research and stochastic control theory. ...
doi:10.1007/978-3-319-91908-9_21
fatcat:yjsuwb5ibjff3cq3niatu6sbxq
Probabilistic Model Checking of Robots Deployed in Extreme Environments
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this paper, we develop a framework for probabilistic model checking on a layered Markov model to verify the safety and reliability requirements of such robots, both at pre-mission stage and during runtime ...
Two novel estimators based on conservative Bayesian inference and imprecise probability model with sets of priors are introduced to learn the unknown transition parameters from operational data. ...
We thank Lorenzo Strigini, Peter Bishop and Andrey Povyakalo from City, University of London who provided insights on the initial ideas of the work. ...
arXiv:1812.04128v3
fatcat:iqw54ooxprcl3mf2x2hfrzsniq
Probabilistic Model Checking of Robots Deployed in Extreme Environments
2019
PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this paper, we develop a framework for probabilistic model checking on a layered Markov model to verify the safety and reliability requirements of such robots, both at pre-mission stage and during runtime ...
Two novel estimators based on conservative Bayesian inference and imprecise probability model with sets of priors are introduced to learn the unknown transition parameters from operational data. ...
We thank Lorenzo Strigini, Peter Bishop and Andrey Povyakalo from City, University of London who provided insights on the initial ideas of the work. ...
doi:10.1609/aaai.v33i01.33018066
fatcat:isoakv3b6rgsvpfksyst2ojhdq
Control Improvisation with Probabilistic Temporal Specifications
2016
2016 IEEE First International Conference on Internet-of-Things Design and Implementation (IoTDI)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We learn from existing data a generative model (for instance, an explicit-duration hidden Markov model, or EDHMM) and then supervise this model in order to guarantee that the generated sequences satisfy ...
some desirable specifications given in Probabilistic Computation Tree Logic (PCTL). ...
Probabilistic Model Checking Our approach relies on the use of a verification method known as probabilistic model checking, which determines if a probabilistic model (such as a Markov chain or Markov decision ...
doi:10.1109/iotdi.2015.33
dblp:conf/iotdi/AkkayaFVDLS16
fatcat:xkajtd2b5vhevielprk3cwnhpi
Automated Verification and Synthesis of Stochastic Hybrid Systems: A Survey
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this survey, we overview the most recent results in the literature and discuss different approaches, including (in)finite abstractions, verification and synthesis for temporal logic specifications, ...
stochastic similarity relations, (control) barrier certificates, compositional techniques, and a selection of results on continuous-time stochastic systems; we finally survey recently developed software ...
survey article provides an introduction to the foundations of SHS, towards an easier understanding of many challenges and existing solutions related to formal verification and control synthesis of these models ...
arXiv:2101.07491v2
fatcat:dpir554ebfclhpj5m7e7fi2hv4
A storm is Coming: A Modern Probabilistic Model Checker
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We launch the new probabilistic model checker storm. It features the analysis of discrete- and continuous-time variants of both Markov chains and MDPs. ...
Experiments on a variety of benchmarks show its competitive performance. ...
Storm supports Markov chains and Markov decision processes (MDPs), both in two forms: discrete time and continuous time. ...
arXiv:1702.04311v1
fatcat:5oazkten7zg4zghea7wqf3s27q
Saturated Path-Constrained MDP: Planning under Uncertainty and Deterministic Model-Checking Constraints
2014
PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
This paper presents Saturated Path-Constrained Markov Decision Processes (SPC MDPs), a new MDP type for planning under uncertainty with deterministic model-checking constraints, e.g., "state s must be ...
In this context, we propose a new problem class, Saturated Path-Constrained Markov Decision Processes (SPC MDPs), specifically designed to marry decision-theoretic planning to model-checking. ...
Introduction Markov Decision Processes (MDPs) are some of the most popular models for optimizing the behavior of stochastic discrete-time dynamical systems. ...
doi:10.1609/aaai.v28i1.9041
fatcat:74vbkkvg55c3tel5s7ku4wbvjm
Survey on learning-based formal methods: Taxonomy, Applications and Possible future directions
2020
IEEE Access
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
INDEX TERMS Formal methods, formal specification, formal verification, learning model, learning specification. 108562 VOLUME 8, 2020 ...
in learning models and/or specifications from observed system behaviors automatically. ...
(DTMC) [18] , Markov decision process (MDP) [18] will not be introduced here due to space limitations, we refer readers to related references for details. ...
doi:10.1109/access.2020.3000907
fatcat:uiy7d2ellrc4jmzn4eumleeoy4
An investigation into serotonergic and environmental interventions against depression in a simulated delayed reward paradigm
[article]
2019
bioRxiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
We show with both standard behavioural simulations and model checking that SSRIs perform significantly better against interventions with psychedelics. ...
for smaller signals but amplifying larger ones. ...
This is achieved in Prism via the use of action labels. Specifically all synchronised transitions have the action label ([timed]). ...
doi:10.1101/580456
fatcat:atqlz66h2fblllj6dfcef4t6bu
« Previous
Showing results 1 — 15 out of 44 results