Filters








14,040 Hits in 6.0 sec

Simplified decision making in the belief space using belief sparsification [article]

Khen Elimelech, Vadim Indelman
2021 arXiv   pre-print
In this work, we introduce a new and efficient solution approach for the problem of decision making under uncertainty, which can be formulated as decision making in a belief space, over a possibly high-dimensional  ...  We then practically apply these ideas to decision problems in the belief space, which can be simplified by considering a sparse approximation of their initial belief.  ...  Andrej Kitanov from the Faculty of Aerospace Engineering at the Technion -Israel Institute of Technology, for insightful discussions concerning Section 3.3.2, and his assistance with implementing the simulation  ... 
arXiv:1909.00885v4 fatcat:gur4uqitdng7zimopzenivv5re

Bayesian sparse sampling for on-line reward optimization

Tao Wang, Daniel Lizotte, Michael Bowling, Dale Schuurmans
2005 Proceedings of the 22nd international conference on Machine learning - ICML '05  
We present an efficient "sparse sampling" technique for approximating Bayes optimal decision making in reinforcement learning, addressing the well known exploration versus exploitation tradeoff.  ...  Our approach combines sparse sampling with Bayesian exploration to achieve improved decision making while controlling computational cost.  ...  Acknowledgments Research supported by the Alberta Ingenuity Centre for Machine Learning, CRC, NSERC, MITACS and CFI.  ... 
doi:10.1145/1102351.1102472 dblp:conf/icml/WangLBS05 fatcat:pwpcdttqufbttklck3qpfxbsua

Robot Task Planning for Low Entropy Belief States [article]

Alphonsus Adu-Bredu and Zhen Zeng and Neha Pusalkar and Odest Chadwicke Jenkins
2020 arXiv   pre-print
We compare FD-Replan with current classical planning and belief space planning approaches by solving low entropy goal-directed grocery packing tasks in simulation.  ...  FD-Replan combines belief space representation with the fast, goal-directed features of classical planning to efficiently plan for low entropy goal-directed reasoning tasks.  ...  As such, belief space planning methods such as POMCP [11] and DESPOT [34] are used to solve POMDPs approximately.  ... 
arXiv:2011.09105v1 fatcat:wifm77yvyzha3iriwrmcesdm6m

Learning is planning: near Bayes-optimal reinforcement learning via Monte-Carlo tree search [article]

John Asmuth, Michael L. Littman
2012 arXiv   pre-print
Recent advances in the use of Monte-Carlo tree search (MCTS) have shown that it is possible to act near-optimally in Markov Decision Processes (MDPs) with very large or infinite state spaces.  ...  FSSS can be used to act efficiently in any possible underlying MDP.  ...  as its oracle) to plan will behave approximately optimally.  ... 
arXiv:1202.3699v1 fatcat:uyptrspdvraankhtgckg34je6y

Intention-aware online POMDP planning for autonomous driving in a crowd

Haoyu Bai, Shaojun Cai, Nan Ye, David Hsu, Wee Sun Lee
2015 2015 IEEE International Conference on Robotics and Automation (ICRA)  
A key feature of our approach is to use the partially observable Markov decision process (POMDP) for systematic, robust decision making under uncertainty.  ...  This indicates that POMDP planning is improving fast in computational efficiency and becoming increasingly practical as a tool for robot planning under uncertainty. The authors are with the  ...  The trade-off between the optimality of decision making and computational efficiency remains, but the gap is narrowing. III.  ... 
doi:10.1109/icra.2015.7139219 dblp:conf/icra/BaiCYHL15 fatcat:7cm5hcotq5ehhiih2cq6ve3diy

Bayesian Reinforcement Learning via Deep, Sparse Sampling [article]

Divya Grover, Debabrota Basu, Christos Dimitrakakis
2020 arXiv   pre-print
The main novelty is the use of a candidate policy generator, to generate long-term options in the planning tree (over beliefs), which allows us to create much sparser and deeper trees.  ...  We address the problem of Bayesian reinforcement learning using efficient model-based online planning.  ...  BAMCP takes a Monte-Carlo approach to sparse lookahead in belief-augmented version of Markov decision process.  ... 
arXiv:1902.02661v4 fatcat:bwofz6wmjfamdm7avnwkrzcfm4

A Model-Based, Decision-Theoretic Perspective on Automated Cyber Response [article]

Lashon B. Booker, Scott A. Musman
2020 arXiv   pre-print
Cyber-attacks can occur at machine speeds that are far too fast for human-in-the-loop (or sometimes on-the-loop) decision making to be a viable option.  ...  responses can be aligned with risk-aware cost/benefit tradeoffs that are defined by user-supplied preferences that capture the key aspects of how human operators understand the system, the adversary and  ...  Belief states are efficiently approximated and updated using particle filters.  ... 
arXiv:2002.08957v1 fatcat:rakhpiufdve4rorxi5o45rhlqy

Information-Guided Robotic Maximum Seek-and-Sample in Partially Observable Continuous Environments

Victoria Preston, Genevieve Flaspohler, Anna Pauline Miranda Michel, Yogesh Girdhar, Nicholas Roy
2019 IEEE Robotics and Automation Letters  
We formulate the MSS problem as a partially-observable Markov decision process (POMDP) with continuous state and observation spaces, and a sparse reward signal.  ...  To solve the MSS POMDP, PLUMES uses an information-theoretic reward heuristic with continous-observation Monte Carlo Tree Search to efficiently localize and sample from the global maximum.  ...  Additionally, we thank the SLI group, RRG and WARPlab for their insight and support. This project was supported by an NSF-GRFP award (G.F.), NDSEG Fellowship award (V.P.), and NSF NRI Award 1734400.  ... 
doi:10.1109/lra.2019.2929997 fatcat:phbfeq6rmvejvim2vzqx2v3dfa

Monte Carlo Tree Search for Bayesian Reinforcement Learning

Ngo Anh Vien, Wolfgang Ertel
2012 2012 11th International Conference on Machine Learning and Applications  
In addition, we examine the use of a parameter tying method to keep the model search space small, and propose the use of nested mixture of tied models to increase robustness of the method when our prior  ...  Bayesian model-based reinforcement learning can be formulated as a partially observable Markov decision process (POMDP) to provide a principled framework for optimally balancing exploitation and exploration  ...  POMCP uses Monte-Carlo sampling to overcome the curse of dimensionality during look-ahead planning, and it uses particle filters to approximate beliefs in very large or continuous state spaces.  ... 
doi:10.1109/icmla.2012.30 dblp:conf/icmla/VienE12 fatcat:au3g7ijkz5cgzoshg2mep6vu34

Monte-Carlo tree search for Bayesian reinforcement learning

Ngo Anh Vien, Wolfgang Ertel, Viet-Hung Dang, TaeChoong Chung
2013 Applied intelligence (Boston)  
In addition, we examine the use of a parameter tying method to keep the model search space small, and propose the use of nested mixture of tied models to increase robustness of the method when our prior  ...  Bayesian model-based reinforcement learning can be formulated as a partially observable Markov decision process (POMDP) to provide a principled framework for optimally balancing exploitation and exploration  ...  POMCP uses Monte-Carlo sampling to overcome the curse of dimensionality during look-ahead planning, and it uses particle filters to approximate beliefs in very large or continuous state spaces.  ... 
doi:10.1007/s10489-012-0416-2 fatcat:3p3duvqfsrcopo5im3q3gbdpuq

Partially Observable Markov Decision Processes [chapter]

Thomas Zeugmann, Pascal Poupart, James Kennedy, Xin Jin, Jiawei Han, Lorenza Saitta, Michele Sebag, Jan Peters, J. Andrew Bagnell, Walter Daelemans, Geoffrey I. Webb, Kai Ming Ting (+12 others)
2011 Encyclopedia of Machine Learning  
belief space is small, efficient approximation possible -If belief reachable when acting optimally is small, small policy exists Characterizing Belief SpaceBelief space size can grow exponentially  ...  , and -branch and bound techniques to eliminate unnecessary search • But these will not be effective if there is no small policy Small Reachable Belief Space • POMDP can be efficiently approximated  ... 
doi:10.1007/978-0-387-30164-8_629 fatcat:hj6hnbjtn5fshpq4jnxpsaj7dq

Reinforcement Learning through Active Inference [article]

Alexander Tschantz, Beren Millidge, Anil K. Seth, Christopher L. Buckley
2020 arXiv   pre-print
Inspired by active inference, we develop and implement a novel objective for decision making, which we term the free energy of the expected future.  ...  We demonstrate that the resulting algorithm successfully balances exploration and exploitation, simultaneously achieving robust performance on several challenging RL benchmarks with sparse, well-shaped  ...  Mortimer and Theresa Sackler Foundation and the School of Engineering and Informatics at the University of Sussex. BM is supported by an EPSRC funded PhDS Studentship.  ... 
arXiv:2002.12636v1 fatcat:c7r63kvapzd4ja5vct5v6ub3ci

Topological Information-Theoretic Belief Space Planning with Optimality Guarantees [article]

Andrej Kitanov, Vadim Indelman
2019 arXiv   pre-print
Our recently introduced topological belief space planning t-bsp instead performs decision making considering only topologies of factor graphs that correspond to posterior future beliefs.  ...  Determining a globally optimal solution of belief space planning (BSP) in high-dimensional state spaces is computationally expensive, as it involves belief propagation and objective function evaluation  ...  CONCLUSIONS This paper provides theoretical foundations for action consistent topological belief space planning (t-BSP). t-BSP enables efficient decision making in high dimensional state spaces by considering  ... 
arXiv:1903.00927v1 fatcat:22eioiahwng7rn7kib7clc2smy

A decision-theoretic formalism for belief-optimal reasoning

Kris Hauser
2009 Proceedings of the 9th Workshop on Performance Metrics for Intelligent Systems - PerMIS '09  
The system is modeled a decision-making agent that moves in a probabilistic belief space, where each information-gathering or computation step changes the belief state.  ...  This forms a Markov decision process (MDP), and the belief-optimal system operates according to the belief-space policy that optimizes the MDP.  ...  POMDP approximation in large belief spaces is an area of active research, and perhaps the most promising results have come from sparse sampling and depth-limited search techniques [12, 13] .  ... 
doi:10.1145/1865909.1865964 dblp:conf/permis/Hauser09 fatcat:cft7vyc5l5cr5lhbnnbodlrz2q

Distributed sensor resource management and planning

Deepak Khosla, James Guillochon, Ivan Kadar
2007 Signal Processing, Sensor Fusion, and Target Recognition XVI  
In our previous paper, we introduced a centralized non-myopic planning algorithm, C-SPLAN, that uses sparse sampling to estimate the value of resource assignments.  ...  Sparse sampling is related to Monte Carlo simulation.  ...  The method we use in this paper to shrink the action space is known as sparse planning. Sparse planning considers a chain of actions up to a certain depth time when making a decision.  ... 
doi:10.1117/12.719929 fatcat:ps2caessojbebecwmcyzge5u3u
« Previous Showing results 1 — 15 out of 14,040 results