Filters








555 Hits in 4.2 sec

Convex Optimization for Parameter Synthesis in MDPs [article]

Murat Cubuktepe, Nils Jansen, Sebastian Junges, Joost-Pieter Katoen, Ufuk Topcu
2021 arXiv   pre-print
We formulate the parameter synthesis problem as a quadratically constrained quadratic program (QCQP), which is nonconvex and is NP-hard to solve in general.  ...  The parameter synthesis problem is to compute an instantiation of these unspecified parameters such that the resulting MDP satisfies the temporal logic specification.  ...  CONCLUSION AND FUTURE WORK We studied the applicability of convex optimization for parameter synthesis of parametric Markov decision processes (pMDPs).  ... 
arXiv:2107.00108v1 fatcat:xmbhqovctzamjgojyt62xj6chy

Scenario-Based Verification of Uncertain MDPs [chapter]

Murat Cubuktepe, Nils Jansen, Sebastian Junges, Joost-Pieter Katoen, Ufuk Topcu
2020 Lecture Notes in Computer Science  
convex optimization problem.  ...  The probability distributions for these random parameters are unknown.  ...  Each concrete parameter value is referred to as a scenario in the convex optimization literature [15] .  ... 
doi:10.1007/978-3-030-45190-5_16 pmid:32754724 pmcid:PMC7402411 fatcat:n3goluiytjbedhpseq5vpqscfa

Scenario-Based Verification of Uncertain MDPs [article]

Murat Cubuktepe, Nils Jansen, Sebastian Junges, Joost-Pieter Katoen, Ufuk Topcu
2020 arXiv   pre-print
convex optimization problem.  ...  The probability distributions for these random parameters are unknown.  ...  Each concrete parameter value is referred to as a scenario in the convex optimization literature [15] .  ... 
arXiv:1912.11223v2 fatcat:u2p2b4kwczce7higtuo72uqvuy

Optimal Deceptive and Reference Policies for Supervisory Control [article]

Mustafa O. Karabag, Melkior Ornik, Ufuk Topcu
2019 arXiv   pre-print
We show that the synthesis of deceptive policies has a convex optimization problem formulation, while the synthesis of reference policies requires solving a nonconvex optimization problem.  ...  optimal deceptive policies for such agents.  ...  ACKNOWLEDGMENTS This work was supported in part by ARO W911NF-15-1-0592, DARPA D19AP00004, and DARPA W911NF-16-1-0001.  ... 
arXiv:1902.00590v1 fatcat:yiarbucc7ffond7kd2w4x7uani

Sequential Convex Programming for the Efficient Verification of Parametric MDPs [article]

Murat Cubuktepe, Nils Jansen, Sebastian Junges, Joost-Pieter Katoen, Ivan Papusha, Hasan A. Poonawala, Ufuk Topcu
2017 arXiv   pre-print
Direct applications of the encodings as nonlinear pro- grams are model repair and parameter synthesis. We demonstrate the scalability and quality of our approach by well-known benchmarks  ...  This insight allows for a sequential optimization algorithm to efficiently compute sound but possibly suboptimal solutions. Each stage of this algorithm solves a geometric programming problem.  ...  Conclusion and future work We presented a way to use convex optimization in the field of parameter synthesis for parametric Markov models.  ... 
arXiv:1702.00063v1 fatcat:7eqtbtjcfvbghmf4vljx2i7ipa

Synthesis in pMDPs: A Tale of 1001 Parameters [article]

Murat Cubuktepe, Nils Jansen, Sebastian Junges, Joost-Pieter Katoen, Ufuk Topcu
2018 arXiv   pre-print
An appropriate interplay between CCP solvers and probabilistic model checkers creates a procedure --- realized in the open-source tool PROPhESY --- that solves the synthesis problem for models with thousands  ...  We show that this problem can be formulated as a quadratically-constrained quadratic program (QCQP) and is non-convex in general.  ...  Conclusion and Future Work We presented a new approach to parameter synthesis for pMDPs.  ... 
arXiv:1803.02884v4 fatcat:5j34rtfjo5f4fnjfcdjjl6zshy

Page 6797 of Mathematical Reviews Vol. , Issue 90K [page]

1990 Mathematical Reviews  
Although the two criteria are equivalent for cer- tain classes of MDPs, their feasible and optimal policies differ for many nontrivial problems.  ...  The model of continuous-time Markov decision programming (MDP) is considered in this article. The author proves that there is a unique solution to the optimal equation of continuous-time MDP.  ... 

Least Inferable Policies for Markov Decision Processes [article]

Mustafa O. Karabag and Melkior Ornik and Ufuk Topcu
2018 arXiv   pre-print
We proceed to solve the problem using convex optimization methods.  ...  We consider the task of the agent as a reachability problem in a Markov decision process and study the synthesis of policies that minimize the observer's ability to infer the transition probabilities of  ...  ACKNOWLEDGEMENT This work was supported in part by DARPA W911NF-16-1-0001.  ... 
arXiv:1809.06482v1 fatcat:s3gx72m4pjaldpr2jub2ksig5a

Optimal control in Markov decision processes via distributed optimization

Jie Fu, Shuo Han, Ufuk Topcu
2015 2015 54th IEEE Conference on Decision and Control (CDC)  
Optimal control synthesis in stochastic systems with respect to quantitative temporal logic constraints can be formulated as linear programming problems.  ...  However, centralized synthesis algorithms do not scale to many practical systems. To tackle this issue, we propose a decomposition-based distributed synthesis algorithm.  ...  Distributed optimization: As a prelude to the distributed synthesis method developed in section IV, now we describe the alternating direction method of multipliers (ADMM) [16] for the generic convex  ... 
doi:10.1109/cdc.2015.7403398 dblp:conf/cdc/FuHT15 fatcat:xw6til2k3zgmhi4mqwc2blsmv4

Evolutionary-Guided Synthesis of Verified Pareto-Optimal MDP Policies

Simos Gerasimou, Javier Camara, Radu Calinescu, Naif Alasmari, Faisal Alhwikem, Xinwei Fang
2021 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)  
genetic algorithms to synthesise Pareto-optimal parameter values that define the required MDP policies.  ...  We present a new approach for synthesising Paretooptimal Markov decision process (MDP) policies that satisfy complex combinations of quality-of-service (QoS) software requirements.  ...  ', the UK EPSRC project EP/R026173/1 'Offshore Robotics for Certification of Assets' (through its PRF project COVE), and the Assuring Autonomy International Programme.  ... 
doi:10.1109/ase51524.2021.9678727 fatcat:kquqi7ahyrhtjaj3ux6qmmrjvi

Multi-Objective Controller Synthesis with Uncertain Human Preferences [article]

Shenghui Chen, Kayla Boggess, David Parker, Lu Feng
2022 arXiv   pre-print
In this paper, we formalize the notion of uncertain human preferences and present a novel approach that accounts for this uncertainty in the context of multi-objective controller synthesis for Markov decision  ...  Complex real-world applications of cyber-physical systems give rise to the need for multi-objective controller synthesis, which concerns the problem of computing an optimal controller subject to multiple  ...  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the grant sponsors.  ... 
arXiv:2105.04662v2 fatcat:d3thyrfdqfaabksgfaltdrng4a

Optimal control in Markov decision processes via distributed optimization [article]

Jie Fu, Shuo Han, Ufuk Topcu
2015 arXiv   pre-print
Optimal control synthesis in stochastic systems with respect to quantitative temporal logic constraints can be formulated as linear programming problems.  ...  However, centralized synthesis algorithms do not scale to many practical systems. To tackle this issue, we propose a decomposition-based distributed synthesis algorithm.  ...  Distributed optimization: As a prelude to the distributed synthesis method developed in section IV, now we describe the alternating direction method of multipliers (ADMM) [16] for the generic convex  ... 
arXiv:1503.07189v1 fatcat:g3z2ki6o4bbxrotwrgahb7fro4

Risk-Averse Decision Making Under Uncertainty [article]

Mohamadreza Ahmadi, Ugo Rosolia, Michel D. Ingham, Richard M. Murray, Aaron D. Ames
2021 arXiv   pre-print
For MDPs, we demonstrate that the formulated optimization problems are in the form of difference convex programs (DCPs) and can be solved by the disciplined convex-concave programming (DCCP) framework.  ...  For MDPs, we reformulate the problem into a infsup problem via the Lagrangian framework and propose an optimization-based method to synthesize Markovian policies.  ...  In Section IV, we present the optimization-based method for designing risk-averse policies for MDPs.  ... 
arXiv:2109.04082v1 fatcat:jfwekgip25bcrdidnetyqncosu

Parameter synthesis for probabilistic timed automata using stochastic game abstractions

Aleksandra Jovanović, Marta Kwiatkowska
2018 Theoretical Computer Science  
We propose a method to synthesise optimal values of timing parameters for probabilistic timed automata, in the sense that the probability of reaching some set of states is either maximised or minimised  ...  In the parametric setting, our method is able to determine all the possible maximum (or minimum) reachability probabilities that arise for different values of timing parameters, and yields optimal valuations  ...  to restrict the parameter domain to bounded integers as in [11] . We are currently implementing the algorithm in Prism. Acknowledgments This research is supported by ERC AdG VERIWARE.  ... 
doi:10.1016/j.tcs.2017.05.005 fatcat:a6wbugvjxnaupijcndpsipr644

Parameter Synthesis for Probabilistic Timed Automata Using Stochastic Game Abstractions [chapter]

Aleksandra Jovanović, Marta Kwiatkowska
2014 Lecture Notes in Computer Science  
We propose a method to synthesise optimal values of timing parameters for probabilistic timed automata, in the sense that the probability of reaching some set of states is either maximised or minimised  ...  In the parametric setting, our method is able to determine all the possible maximum (or minimum) reachability probabilities that arise for different values of timing parameters, and yields optimal valuations  ...  to restrict the parameter domain to bounded integers as in [11] . We are currently implementing the algorithm in Prism. Acknowledgments This research is supported by ERC AdG VERIWARE.  ... 
doi:10.1007/978-3-319-11439-2_14 fatcat:jgffahmybjaxpognc2j2fzqgiu
« Previous Showing results 1 — 15 out of 555 results