Filters








3,806 Hits in 3.9 sec

On the Computational Complexity of Stochastic Controller Optimization in POMDPs [article]

Nikos Vlassis, Michael L. Littman, David Barber
2012 arXiv   pre-print
Our result establishes that the more general problem of stochastic controller optimization in POMDPs is also NP-hard.  ...  We show that the problem of finding an optimal stochastic 'blind' controller in a Markov decision process is an NP-hard problem.  ...  Acknowledgments The first author would like to thank Constantinos Daskalakis, Michael Tsatsomeros, John Tsitsiklis, and Steve Vavasis for helpful discussions.  ... 
arXiv:1107.3090v2 fatcat:zy6lbraq2zbjdlqm46a7ldyn6m

On the Computational Complexity of Stochastic Controller Optimization in POMDPs

Nikos Vlassis, Michael L. Littman, David Barber
2012 ACM Transactions on Computation Theory  
Our result establishes that the more general problem of stochastic controller optimization in POMDPs is also NP-hard.  ...  We show that the problem of finding an optimal stochastic "blind" controller in a Markov decision process is an NP-hard problem.  ...  Acknowledgments We are grateful to Marek Petrik for his feedback and for pointing an error in an earlier version.  ... 
doi:10.1145/2382559.2382563 fatcat:k5noabwolnfjpoj7bc4ww5tgc4

Partially Observed Markov Decision Processes: From Filtering to Controlled Sensing [Bookshelf]

Bo Wahlberg
2019 IEEE Control Systems  
It builds on the theory of Markov decision processes (MDPs) and hidden Markov models (HMMs) and results in stochastic optimal control policies that map the estimated state into actions.  ...  POMDPs arise in numerous applications, including optimal search problems, controlled sensing (where sensors adapt their behavior in real time), mobile robotics, multiagent systems, computer vision, active  ... 
doi:10.1109/mcs.2019.2913493 fatcat:2eozixvhbjhzbby27rxrgqh4oa

Periodic Finite State Controllers for Efficient POMDP and DEC-POMDP Planning

Joni Pajarinen, Jaakko Peltonen
2011 Neural Information Processing Systems  
The policy in infinite-horizon POMDP and DEC-POMDP problems has been represented as finite state controllers (FSCs).  ...  We introduce a novel class of periodic FSCs, composed of layers connected only to the previous and next layer.  ...  Acknowledgments We thank Ari Hottinen for discussions on decision making in wireless networks. The authors belong to the Adaptive Informatics Research Centre (CoE of the Academy of Finland).  ... 
dblp:conf/nips/PajarinenP11 fatcat:wwtd3qb4ubgnxdjivrgwu2qjw4

A Novel Single-DBN Generative Model for Optimizing POMDP Controllers by Probabilistic Inference

Igor Kiselev, Pascal Poupart
2014 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
The proposed approaches to policy optimization by probabilistic inference are evaluated on several POMDP benchmark problems and the performance of the implemented approximation algorithms is compared.  ...  POMDP controllers, and (2) we developed several inference approaches to approximate the value of the policy when exact inference methods are not tractable to solve large-size problems with complex graphical  ...  The original task of optimizing POMDP controllers can now be approached by maximizing this objective likelihood of observing reward in a mixture-DBN generative model.  ... 
doi:10.1609/aaai.v28i1.9100 fatcat:qcwoem5sarb6fidqmp6ujgqbqa

Dynamic decision making in stochastic partially observable medical domains: Ischemic heart disease example [chapter]

Milos Hauskrecht
1997 Lecture Notes in Computer Science  
The focus of this paper is the framework of partially observable Markov decision processes (POMDPs) and its role in modeling and solving complex dynamic decision problems in stochastic and partially observable  ...  The paper summarizes some of the basic features of the POMDP framework and explores its potential in solving the problem of the management of the patient with chronic ischemic heart disease.  ...  Peter Szolovits and William Long have provided valuable feedback on early versions of the paper and Hamish Fraser has helped with the ischemic heart disease example.  ... 
doi:10.1007/bfb0029462 fatcat:bk4mlyrgtfcaxjrtlbwzdzlfjy

Tractable Dual Optimal Stochastic Model Predictive Control: An Example in Healthcare [article]

Martin A. Sehr, Robert R. Bitmead
2017 arXiv   pre-print
Output-Feedback Stochastic Model Predictive Control based on Stochastic Optimal Control for nonlinear systems is computationally intractable because of the need to solve a Finite Horizon Stochastic Optimal  ...  In practice, intractability of Stochastic Model Predictive Control is typically overcome by replacement of the underlying Stochastic Optimal Control problem by more amenable approximate surrogate problems  ...  This results in maintaining duality in a stochastic optimal sense, although on the approximate POMDP dynamics. A.  ... 
arXiv:1704.07770v1 fatcat:gmaeiuzddbckxgj63hx334uetm

Stochastic System Monitoring and Control

Gregory M. Provan
2001 International Conference on Artificial Intelligence and Statistics  
Instead of assuming that the process evolves only due to control actions, we assume that system evolution occurs due to both internal system dynamics and control actions, referred to as endogenous and  ...  In this article we propose a new technique for efficiently solving a specialized instance of a finite state sequential decision process.  ...  Control Optimization this task initializes the assumables as evidence and computes the optimal (stochastic) control.  ... 
dblp:conf/aistats/Provan01 fatcat:jga3sufnmfh4nos2tpte4mcpay

Solving POMDPs using quadratically constrained linear programs

Christopher Amato, Daniel S. Bernstein, Shlomo Zilberstein
2006 Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems - AAMAS '06  
One promising approach is based on representing POMDP policies as finite-state controllers. This method has been used successfully to address the intractable memory requirements of POMDP algorithms.  ...  While the optimization algorithms used in this paper only guarantee locally optimal solutions, the results show consistent improvement of solution quality over the state-of-the-art techniques.  ...  As the optimization complexity primarily depends on the controller size and not the size of the POMDP, this allows the algorithm to scale up to large problems.  ... 
doi:10.1145/1160633.1160694 dblp:conf/atal/AmatoBZ06 fatcat:3gvgfex2z5gqrnk363ngwk25di

Optimal Immunization Policy Using Dynamic Programming [article]

Atiye Alaeddini, Daniel Klein
2020 arXiv   pre-print
We apply a stochastic dynamic programming approach to identify the optimal time to change the health intervention policy and the value of decision relevant information for improving the impact of the policy  ...  It is with this understanding that we pursue greater clarity on, and methods to address optimal policy making in health.  ...  Given a stochastic control problem for a POMDP in which there are a number of observation options available to us, with varying associated costs.  ... 
arXiv:1910.08677v2 fatcat:h2fht3yppnbwjcqrkqwgycqxse

Partially observable markov decision processes for artificial intelligence [chapter]

Leslie Pack Kaelbling, Michael L. Littman, Anthony R. Cassandra
1996 Lecture Notes in Computer Science  
In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains.  ...  We then outline a novel algorithm for solving POMDPs off line and show how, in many cases, a finite-memory controller can be extracted from the solution to a POMDP. We conclude with a simple example.  ...  Introduction In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains.  ... 
doi:10.1007/bfb0013957 fatcat:hxrke3yr2nejpj4qyvyt7oaere

Resolving the measurement uncertainty paradox in ecological management [article]

Milad Memarzadeh, Carl Boettiger
2018 arXiv   pre-print
Ecological management and decision-making typically focus on uncertainty about the future, but surprisingly little is known about how to account for uncertainty of the present: that is, the realities of  ...  Our primary paradigms for handling decisions under uncertainty -- the precautionary principle and optimal control -- have so far given contradictory results.  ...  CB also acknowledges computational resources from NSF's XSEDE Jetstream (DEB160003) and Chameleon cloud platforms, as well as the support by the USDA Hatch project CA-B-INS-0162-H.  ... 
arXiv:1812.11184v1 fatcat:xiftr76fgvffbevt5yh7anfgsm

Decentralized control of partially observable Markov decision processes

Christopher Amato, Girish Chowdhary, Alborz Geramifard, N. Kemal Ure, Mykel J. Kochenderfer
2013 52nd IEEE Conference on Decision and Control  
We describe the frameworks, along with the complexity of optimal control and important properties.  ...  This paper surveys recent work on decentralized control of MDPs in which control of each agent depends on a partial view of the world.  ...  In Section IV we present the computational complexity of the Dec-POMDP and a number of subclasses.  ... 
doi:10.1109/cdc.2013.6760239 dblp:conf/cdc/AmatoCGUK13 fatcat:fe5yksf4zjfnrmhswb2rspipji

Receding horizon stochastic control algorithms for sensor management

Darin Hitchings, David A Castan
2010 Proceedings of the 2010 American Control Conference  
While such problems can usually be formulated as stochastic control problems, the resulting optimization problems are complex and difficult to solve in real-time applications.  ...  We propose alternative approaches for sensor management based on receding horizon control using a stochastic control approximation to the sensor management problem.  ...  In terms of the computational complexity of our RH algorithms, the main bottleneck is the solution of the POMDP problems.  ... 
doi:10.1109/acc.2010.5531634 fatcat:snbzkhstivax7dqdazzifca74y

A framework of stochastic power management using hidden Markov model

Ying Tan, Qinru Qiu
2008 Proceedings of the conference on Design, automation and test in Europe - DATE '08  
Compared with traditional optimization technique, which is based on value iteration, the QCLP based optimization provides superior policy by enabling stochastic control.  ...  The effectiveness of stochastic power management relies on the accurate system and workload model and effective policy optimization.  ...  It is believed that the optimization complexity of QCLP problems primarily depends on the controller size, not the size of the POMDP [11] , so this QCLP algorithm can be used to solve POMDP models for  ... 
doi:10.1145/1403375.1403402 fatcat:ebgup6x6yjdrbmzsvv6be2r2ta
« Previous Showing results 1 — 15 out of 3,806 results