Filters








13,066 Hits in 8.4 sec

Partial and Conditional Expectations in Markov Decision Processes with Integer Weights [article]

Jakob Piribauer, Christel Baier
2019 arXiv   pre-print
The paper addresses two variants of the stochastic shortest path problem ('optimize the accumulated weight until reaching a goal state') in Markov decision processes (MDPs) with integer weights.  ...  There are polynomial-time algorithms to check the finiteness of the supremum of the partial or conditional expectations in MDPs with arbitrary integer weights.  ...  Appendix A Partial and Conditional Expectations in Markov Decision Processes In this section, we give proofs to the claims of Section 3.  ... 
arXiv:1902.04538v2 fatcat:mflhfpcbjrgmda34qvjxeanvm4

Partial and Conditional Expectations in Markov Decision Processes with Integer Weights [chapter]

Jakob Piribauer, Christel Baier
2019 Green Chemistry and Sustainable Technology  
The paper addresses two variants of the stochastic shortest path problem ("optimize the accumulated weight until reaching a goal state") in Markov decision processes (MDPs) with integer weights.  ...  There are polynomial-time algorithms to check the finiteness of the supremum of the partial or conditional expectations in MDPs with arbitrary integer weights.  ...  The authors are supported by the DFG through the Research Training Group QuantLA (GRK 1763), the DFG-project BA-1679/11-1, the Collaborative Research Center HAEC (SFB 912), and the cluster of excellence  ... 
doi:10.1007/978-3-030-17127-8_25 dblp:conf/fossacs/PiribauerB19 fatcat:a2j43g4lrjefxoiubjneuhwd2m

On Skolem-Hardness and Saturation Points in Markov Decision Processes

Jakob Piribauer, Christel Baier, Emanuela Merelli, Anuj Dawar, Artur Czumaj
2020 International Colloquium on Automata, Languages and Programming  
In this paper, the inherent mathematical difficulty of a series of optimization problems on Markov decision processes (MDPs) is shown by a reduction from the Positivity problem to the associated decision  ...  The optimization problems under consideration are two non-classical variants of the stochastic shortest path problem (SSPP) in terms of expected partial or conditional accumulated weights, the optimization  ...  Notations for Markov decision processes.  ... 
doi:10.4230/lipics.icalp.2020.138 dblp:conf/icalp/PiribauerB20 fatcat:lxuhfgvfx5bgzphfk4sf5qtsem

Page 7808 of Mathematical Reviews Vol. , Issue 95m [page]

1995 Mathematical Reviews  
This paper considers a semi-Markov decision process X(t) in which, at time ¢, if the system is in state x, and action a is taken, there is an expected cost c(x,a) and a state probability transition law  ...  {For the entire collection see MR 95j:60004. } Douglas John White (1-VA-SE; Charlottesville, VA) 95m:90148 90C40 90C29 Wakuta, Kazuyoshi (J-NAGTC; Nagaoka) Vector-valued Markov decision processes and the  ... 

Page 3290 of Mathematical Reviews Vol. , Issue 91F [page]

1991 Mathematical Reviews  
Connections with partial balance and the structure of traffic equations are worked out. The results are applied to queueing networks with batch ser- vice and arrivals and clustering processes.  ...  This paper deals with Markov processes with interventions.  ... 

The existence of good Markov strategies for decision processes with general payoffs

Theodore P. Hill, Victor C. Pestien
1987 Stochastic Processes and their Applications  
This cl;~cs includes product and lim inf rewards, as well as prxtic ;dly all the classical dynamic programming expected pnyolf function.\. reward function as well, in contrast to the usual randomized Markov  ...  For coun~ahlc-stale decision processes (dynamic programming problems). a general class of objective functions is identified for which it is shown that good blarkov strategies alway exist.  ...  Acknowledgement The authors are grateful to the referees for several useful suggestions and comments.  ... 
doi:10.1016/0304-4149(87)90028-7 fatcat:pzaccbhuc5dwdgqxvyqp56mmva

On Skolem-hardness and saturation points in Markov decision processes [article]

Jakob Piribauer, Christel Baier
2020 arXiv   pre-print
In this paper, the inherent mathematical difficulty of a series of optimization problems on Markov decision processes (MDPs) is shown by a reduction from the Positivity problem to the associated decision  ...  The optimization problems under consideration are two non-classical variants of the stochastic shortest path problem (SSPP) in terms of expected partial or conditional accumulated weights, the optimization  ...  Optimizing the expected mean payoff in energy Markov decision processes. In 14th International Symposium on Automated Technology for 13 P. Cerný, T. A. Henzinger, and A. Radhakrishna.  ... 
arXiv:2004.11441v1 fatcat:qeg2epgmjrhnnch2kylot4sqoe

Properties of a job search problem on a partially observable Markov chain in a dynamic economy

Tōru Nakai
2006 Computers and Mathematics with Applications  
observable Markov process.  ...  The total positivity of order two, or simply TP2, is a fundamental property to investigate sequential decision problems, and it also plays an important role in the Bayesian learning procedure for a partially  ...  It is possible to generalize this order relation in order to investigate a partially observable Markov process, and the details regarding this process are shown in [5, 9] with the applications to the  ... 
doi:10.1016/j.camwa.2005.11.019 fatcat:vetagfnwsbhmlg62upjgri6byi

Information state for Markov decision processes with network delays

Sachin Adlakha, Sanjay Lall, Andrea Goldsmith
2008 2008 47th IEEE Conference on Decision and Control  
Such a networked Markov decision process with delays can be represented as a partially observed Markov decision process (POMDP).  ...  This result generalizes the previous results on Markov decision processes with delayed state information. 2 S. Lall is with the  ...  Model and Definitions Markov Decision Processes A Markov decision process provides a framework for sequential decision making in a stochastic environment.  ... 
doi:10.1109/cdc.2008.4739234 dblp:conf/cdc/AdlakhaLG08 fatcat:avijpx4ehrfo5i7bzioksg3pqe

Page 2199 of Mathematical Reviews Vol. , Issue 99c [page]

1991 Mathematical Reviews  
In the present book, after a brief review of optimization algorithms and their stochastic counterparts, in Chapter 2, Markov chains and Markov processes are presented, where much emphasis is put on the  ...  with arbitrary weights (101-114); P.  ... 

High-Voltage Cable Condition Assessment Method Based on Multi-Source Data Analysis

Xiao-Kai Meng, Yan-Bing Jia, Zhi-Heng Liu, Zhi-Qiang Yu, Pei-Jie Han, Zhu-Mao Lu, Tao Jin
2022 Energies  
a method based on the weight space Markov chain and Monte Carlo method (Markov chains Monte Carlo, MCMC) is proposed.  ...  In view of the problem that the weight value given by the previous state evaluation method is fixed and single and cannot analyze the influence of the weight vector deviation on the evaluation result,  ...  In the random process, the conditional probability can satisfy the following formula within any integer n ∈ T and any i 0 , i 1 , . . . , i n+1 ∈ I.  ... 
doi:10.3390/en15041369 fatcat:qtlrqvbdong67egy5xltnpuf2q

Optimal Properties of Stimulus—Response Learning Models

Aldo Rustichini
1999 Games and Economic Behavior  
The main conclusion of the paper is that linear procedures always converge to optimal action in the case of partial information, and do not in the case of full information.  ...  We consider stimulus-response models of learning: an agent chooses an action with a probability, which is increasing in the payo® that has been associated to that action in past periods.  ...  But the process tends to the optimal action, while with the linear procedures and partial information the process converges to the optimal point only if the initial condition is right.  ... 
doi:10.1006/game.1999.0712 fatcat:lv7b4nirzregpaway6dblvngs4

Resilience Enhancement With Sequentially Proactive Operation Strategies

Chong Wang, Yunhe Hou, Feng Qiu, Shunbo Lei, Kai Liu
2017 IEEE Transactions on Power Systems  
The uncertain sequential transition of system states driven by the evolution of extreme events is modeled as a Markov process.  ...  state as the weight of each objective.  ...  Chen-Ching Liu with Washington State University for insightful suggestions and helpful discussions.  ... 
doi:10.1109/tpwrs.2016.2622858 fatcat:u3bkebhaubfkrl3d7vdim6r7qe

Pseudo Expectation: A Tool for Analyzing Local Search Algorithms

Osamu Watanabe
2005 Progress of Theoretical Physics Supplement  
In [WST03] , the notion of pseudo expectation has been proposed for analyzing relatively simple Markov processes, which would be often seen as simple execution models of local search algorithms.  ...  In this paper, we first explain how it is used, and then investigate the approximation error bound of pseudo expectations.  ...  Tanaka, and R. Monasson, for their interest to this work and various useful suggestions. In particular, I thank to K. Tanaka for teaching me the image restoration algorithm, and to R.  ... 
doi:10.1143/ptps.157.338 fatcat:c3oir3r36jfw5pxdofkm6goc5y

Optimal control and optimal sensor activation for Markov decision problems with costly observations

Rene K. Boel, Jan H. van Schuppen
2015 2015 IEEE Conference on Control Applications (CCA)  
This paper considers partial observation Markov decision processes.  ...  A new concept of pinned conditional distributions of the state given the observed history of the plant is required in order to implement these control laws online.  ...  Partially observed Markov decision processes The example treated in section 2 was extremely simplified.  ... 
doi:10.1109/cca.2015.7320814 dblp:conf/IEEEcca/BoelS15 fatcat:jwycgezndbf7zledu6wifo2qay
« Previous Showing results 1 — 15 out of 13,066 results