Filters








15,359 Hits in 5.4 sec

Complexity of finite-horizon Markov decision process problems

Martin Mundhenk, Judy Goldsmith, Christopher Lusena, Eric Allender
2000 Journal of the ACM  
of Finite-Horizon Markov Decision Process Problems 21 that FP P FPSPACE.  ...  It seems that this process requires both negative and positive rewards. Complexity of Finite-Horizon Markov Decision Process Problems 5 for the rewards.  ... 
doi:10.1145/347476.347480 fatcat:dko66dgsi5cfvph4uzmtkyppqm

Page 6542 of Mathematical Reviews Vol. , Issue 99i [page]

1999 Mathematical Reviews  
For non-homogeneous Markov decision processes with finite state and action spaces, and with rewards and costs, the problem of maximizing the average expected reward under a constraint on the expected discounted  ...  ISBN 0-471-16120-9 This book presents an introduction to Markov decision chains (MDCs), also known as Markov decision processes, Markov con- trol processes, or stochastic dynamic programs.  ... 

Markov Decision Processes: A Tool for Sequential Decision Making under Uncertainty

Oguzhan Alagoz, Heather Hsu, Andrew J. Schaefer, Mark S. Roberts
2009 Medical decision making  
Markov decision processes generalize standard Markov models in that a decision process is embedded in the model and multiple decisions are made over time.  ...  We provide a tutorial on the construction and evaluation of Markov decision processes (MDPs), which are powerful analytical tools used for sequential decision making under uncertainty that have been widely  ...  ACKNOWLEDGMENTS Supported through National Science Foundation grants CMII-0700094 and CMMI-0546960 and by National Library of Medicine grant R21-LM008273.  ... 
doi:10.1177/0272989x09353194 pmid:20044582 pmcid:PMC3060044 fatcat:kgtaq5ymzzhpxovqygxosu7ame

Dynamic decision making in stochastic partially observable medical domains: Ischemic heart disease example [chapter]

Milos Hauskrecht
1997 Lecture Notes in Computer Science  
The focus of this paper is the framework of partially observable Markov decision processes (POMDPs) and its role in modeling and solving complex dynamic decision problems in stochastic and partially observable  ...  The paper summarizes some of the basic features of the POMDP framework and explores its potential in solving the problem of the management of the patient with chronic ischemic heart disease.  ...  This research was supported by the grant 1T]5LM07092 from the National Library of Medicine.  ... 
doi:10.1007/bfb0029462 fatcat:bk4mlyrgtfcaxjrtlbwzdzlfjy

Optimal Intervention in Semi-Markov-Based Asynchronous Probabilistic Boolean Networks

Qiuli Liu, Qingguo Zeng, Jinghao Huang, Deliang Li
2018 Complexity  
This paper attempts to study an optimal control problem in a generalized asynchronous PBN by employing the theory of average value-at-risk (AVaR) for finite horizon semi-Markov decision processes.  ...  Specifically, we first formulate a control model for a generalized asynchronous PBN as an AVaR model for finite horizon semi-Markov decision processes and then solve an optimal control problem for minimizing  ...  Therefore, our optimality problem is actually described as an AVaR model in finite horizon semi-Markov decision processes, and the optimality technique developed well in semi-Markov decision processes  ... 
doi:10.1155/2018/8983670 fatcat:4zcgzt5ddbeb3kydxt3k2y4fui

A Survey of Applications of Markov Decision Processes

D. J. White
1993 Journal of the Operational Research Society  
Applications of Markov decision processes Short summary of the problem D. J. White—A Survey of Applications of Markov Decision Processes Objective function Comments 1.  ...  White—A Survey of Applications of Markov Decision Processes Comments The problem is modelled as an infinite-horizon stochastic dynamic program.  ... 
doi:10.1057/jors.1993.181 fatcat:56fg3qgevrfw7fy2oni7fdx4hi

A Survey of Applications of Markov Decision Processes

D J White
1993 Journal of the Operational Research Society  
Applications of Markov decision processes Short summary of the problem D. J. White—A Survey of Applications of Markov Decision Processes Objective function Comments 1.  ...  White—A Survey of Applications of Markov Decision Processes Comments The problem is modelled as an infinite-horizon stochastic dynamic program.  ... 
doi:10.1038/sj/jors/0441103 fatcat:az64bkbn7zdhhjhhcjmq2zpe5i

Page 5046 of Mathematical Reviews Vol. , Issue 2004f [page]

2004 Mathematical Reviews  
Summary: “We propose for risk sensitive control of finite Markov chains a counterpart of the popular ‘actor-critic’ algorithm for classical Markov decision processes.  ...  ; Seoul) ; Marcus, Steven I. (1-MD-ECE; College Park, MD) Approximation receding horizon approach for Markov decision processes: average reward case.  ... 

Page 6797 of Mathematical Reviews Vol. , Issue 90K [page]

1990 Mathematical Reviews  
Summary: “This paper develops a new framework for the study of Markov decision processes in which the control problem is viewed as an optimization problem on the set of canonically induced mea- sures on  ...  .; Nguen Ngok Tkhang A parametric problem in a model of Markov decision-making processes. (Russian) Vestnik Khar'kov. Gos. Univ. No. 315 Upravl. Sistemy (1988), 35-40.  ... 

The Complexity of Decentralized Control of Markov Decision Processes [article]

Daniel S Bernstein, Shlomo Zilberstein, Neil Immerman
2013 arXiv   pre-print
These complexity results illustrate a fundamental difference between centralized and decentralized control of Markov processes.  ...  For even a small number of agents, the finite-horizon problems corresponding to both of our models are complete for nondeterministic exponential time.  ...  We discuss the computational complexity of finding opti mal policies for the finite-horizon versions of these prob lems.  ... 
arXiv:1301.3836v1 fatcat:rawrt4esirfm7m2h6vnpu2aae4

Page 6364 of Mathematical Reviews Vol. , Issue 95j [page]

1995 Mathematical Reviews  
processes and finite approximations.  ...  Under these assumptions a theorem is proved which shows that the solution to the finite horizon optimization problem con- verges in a Cesaro sense to the solution of the corresponding infinite horizon  ... 

Page 8704 of Mathematical Reviews Vol. , Issue 2002K [page]

2002 Mathematical Reviews  
[Yin, Gang George] (1-WYNS; Detroit, MI) Singularly perturbed Markov decision processes with inclusion of transient states. (English summary) J. Syst. Sci. Complex. 14 (2001), no. 2, 199-211.  ...  This is then a stan- dard finite horizon problem, and existence of optimal policies and algorithms for their computation are given.  ... 

The Complexity of Decentralized Control of Markov Decision Processes

Daniel S. Bernstein, Robert Givan, Neil Immerman, Shlomo Zilberstein
2002 Mathematics of Operations Research  
These complexity results illustrate a fundamental difference between centralized and decentralized control of Markov processes.  ...  For even a small number of agents, the finite-horizon problems corresponding to both of our models are complete for nondeterministic exponential time.  ...  We discuss the computational complexity of finding optimal policies for the finite-horizon versions of these problems.  ... 
doi:10.1287/moor.27.4.819.297 fatcat:aycva7j6urdbhofd4534flpchq

LEARNING-BASED MODEL PREDICTIVE CONTROL FOR MARKOV DECISION PROCESSES

Rudy R. Negenborn, Bart De Schutter, Marco A. Wiering, Hans Hellendoorn
2005 IFAC Proceedings Volumes  
We propose the use of Model Predictive Control (MPC) for controlling systems described by Markov decision processes. First, we consider a straightforward MPC algorithm for Markov decision processes.  ...  This speeds up the decision making, allows decisions to be made over an infinite instead of a finite horizon, and provides adequate control actions, even if the system and desired performance slowly vary  ...  HYbrid CONtrol: Taming Heterogeneity and Complexity of Networked Embedded Systems (HYCON)".  ... 
doi:10.3182/20050703-6-cz-1902.00280 fatcat:6v7vdwb5rnhjnm6eccoz2hxfwm

Page 586 of Mathematical Reviews Vol. , Issue 95a [page]

1995 Mathematical Reviews  
A discounted Markov decision process with finite state space and infinite horizon is considered.  ...  This paper presents a new policy iteration scheme for Markov decision processes (MDP) with finite state space and action space.  ... 
« Previous Showing results 1 — 15 out of 15,359 results