469 Hits in 2.4 sec

Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning [article]

Nat Dilokthanakul, Christos Kaplanis, Nick Pawlowski, Murray Shanahan
2017 arXiv   pre-print
Hierarchical reinforcement learning (HRL) tackles this problem by using a set of temporally-extended actions, or options, each of which has its own subgoal.  ...  We incorporate such subgoals in an end-to-end hierarchical reinforcement learning system and test two variants of our algorithm on a number of games from the Atari suite.  ...  There are a large number of works on subgoal discovery [27, 19, 18] , most of which are based on finding bottleneck states.  ... 
arXiv:1705.06769v2 fatcat:hazlih23wfgrdfnupmec2etvpi

Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation [article]

Yifeng Zhu, Peter Stone, Yuke Zhu
2022 arXiv   pre-print
Our method has shown superior performance over state-of-the-art imitation learning methods in multi-stage manipulation tasks.  ...  We tackle real-world long-horizon robot manipulation tasks through skill discovery.  ...  Notable ones include the options framework [2, 10, 13, 23, 26, 50] and unsupervised skill discovery based on information-theoretic metrics [8, 15, 43] .  ... 
arXiv:2109.13841v2 fatcat:6bxaf5ajwjapvir4vrosmfymgu

Autonomously constructing hierarchical task networks for planning and human-robot collaboration

Bradley Hayes, Brian Scassellati
2016 2016 IEEE International Conference on Robotics and Automation (ICRA)  
We present evaluations within a multi-resolution goal inference task and a transfer learning application showing the utility of our approach.  ...  a wide variety of use cases critical to human-robot interaction.  ...  Adapting arbitrary primitive actions to serve as subgoal specifications may involve utilizing a HI-MAT hierarchy [25] or learning options that achieve the 'bottleneck states' discovered via a state space  ... 
doi:10.1109/icra.2016.7487760 dblp:conf/icra/HayesS16 fatcat:4hvpcix6o5daxpur4jgkgbl2ky

Hierarchical Reinforcement Learning: A Survey and Open Research Challenges

Matthias Hutsebaut-Buysse, Kevin Mets, Steven Latré
2022 Machine Learning and Knowledge Extraction  
We then introduce the Options framework, which provides a more generic approach, allowing abstractions to be discovered and learned semi-automatically.  ...  In order to further advance the development of HRL agents, capable of simultaneously learning abstractions and how to use them, solely from interaction with complex high dimensional environments, we also  ...  Options as a Tool for Meta-Learning In a meta-learning approach [214] [215] [216] , we search for adaptability.  ... 
doi:10.3390/make4010009 fatcat:emexhacqtvgdbelvbufusneira

Visual recipes for slicing and dicing data: teaching data wrangling using subgoal graphics

Lovisa Sundin, Nourhan Sakr, Juho Leinonen, Sherif Aly, Quintin Cutts
2021 21st Koli Calling International Conference on Computing Education Research  
An example of subgoal graphics visualizing how to access the matrix diagonal, here without subgoal labels.  ...  Non-majors often need to learn it quickly alongside their other subjects. Previous research suggests that subgoal labels offer a powerful scaffolding strategy to help novices decompose problems.  ...  ACKNOWLEDGMENTS Our data collection was possible with the help of Eric Yao, Junaid Akhtar, Briana Morrison,, and Syed Waqar Nabi.  ... 
doi:10.1145/3488042.3488063 fatcat:ai4f44neyjcjtbogwotubjb2eu

Multi-Level Discovery of Deep Options [article]

Roy Fox, Sanjay Krishnan, Ion Stoica, Ken Goldberg
2017 arXiv   pre-print
The scalability of our approach to multi-level hierarchies stems from the decoupling of low-level option discovery from high-level meta-control policy learning, facilitated by under-parametrization of  ...  We present Discovery of Deep Options (DDO), a policy-gradient algorithm that discovers parametrized options from a set of demonstration trajectories, and can be used recursively to discover additional  ...  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Sponsors.  ... 
arXiv:1703.08294v2 fatcat:voytyqqx5vhiliflgay5vge3xi

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation [article]

Tejas D. Kulkarni, Karthik R. Narasimhan, Ardavan Saeedi, Joshua B. Tenenbaum
2016 arXiv   pre-print
We demonstrate the strength of our approach on two problems with very sparse, delayed feedback: (1) a complex discrete stochastic decision process, and (2) the classic ATARI game 'Montezuma's Revenge'.  ...  Learning goal-directed behavior in environments with sparse feedback is a major challenge for reinforcement learning algorithms.  ...  In the context of hierarchical RL, Goel and Huber [13] discuss a framework for subgoal discovery using the structural aspects of a learned policy model. Şimşek et al.  ... 
arXiv:1604.06057v2 fatcat:p33suojusrcpfpg4ybc4hrfj6y

Induction and Exploitation of Subgoal Automata for Reinforcement Learning

Daniel Furelos-Blanco, Mark Law, Anders Jonsson, Krysia Broda, Alessandra Russo
2021 The Journal of Artificial Intelligence Research  
ISA interleaves reinforcement learning with the induction of a subgoal automaton, an automaton whose edges are labeled by the task's subgoals expressed as propositional logic formulas over a set of high-level  ...  A state-of-the-art inductive logic programming system is used to learn a subgoal automaton that covers the traces of high-level events observed by the RL agent.  ...  It interleaves option discovery and learning of policies for the discovered options.  ... 
doi:10.1613/jair.1.12372 fatcat:yxtefk4vbjam7mtmtbs4bo2lcq

Hierarchical principles of embodied reinforcement learning: A review [article]

Manfred Eppe, Christian Gumbsch, Matthias Kerzel, Phuong D.H. Nguyen, Martin V. Butz, Stefan Wermter
2020 arXiv   pre-print
We expect our results to guide the development of more sophisticated cognitively inspired hierarchical methods, so that future artificial agents achieve a problem-solving performance on the level of intelligent  ...  Among the most promising computational approaches to provide comparable learning-based problem-solving abilities for artificial agents and robots is hierarchical reinforcement learning.  ...  An alternative method to model curiosity is to perform hypothesis-testing for option discovery 16 .  ... 
arXiv:2012.10147v1 fatcat:dfkdehyz2rggtimmlcmtvycpxe

Reinforcement Learning and its Connections with Neuroscience and Psychology [article]

Ajay Subramanian, Sharad Chitlangia, Veeky Baths
2021 arXiv   pre-print
In this paper, we comprehensively review a large number of findings in both neuroscience and psychology that evidence reinforcement learning as a promising candidate for modeling learning and decision  ...  In doing so, we construct a mapping between various classes of modern RL algorithms and specific findings in both neurophysiological and behavioral literature.  ...  One approach to option discovery is to keep a record of states that occur frequently on paths to goals and label them as subgoals or bottleneck states that a good solution must pass through [203, 204,  ... 
arXiv:2007.01099v5 fatcat:mjpkztlmqnfjba3dtcwqwmmlvu

Discovery of hierarchical representations for efficient planning

Momchil S. Tomov, Samyukta Yagati, Agni Kumar, Wanqian Yang, Samuel J. Gershman, David Pascucci
2020 PLoS Computational Biology  
Building on principles developed in structure learning and robotics, the model predicts that hierarchy discovery should be sensitive to the topological structure, reward distribution, and distribution  ...  We then test the novel predictions of the model in eight behavioral experiments, demonstrating how the distribution of tasks and rewards can influence planning behavior via the discovered hierarchy, sometimes  ...  Edges in the highlevel graph H can be seen as options, with subgoals specified by the endpoints of bridges and option policies specifying how to reach the subgoals within a cluster.  ... 
doi:10.1371/journal.pcbi.1007594 pmid:32251444 fatcat:thfpisii6ve3vneqdfjwfk2zp4

Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery [article]

Jiachen Yang, Igor Borovikov, Hongyuan Zha
2020 arXiv   pre-print
Agents learn useful and distinct skills at the low level via independent Q-learning, while they learn to select complementary latent skill variables at the high level via centralized multi-agent training  ...  unsupervised skill discovery.  ...  to improve the clarity and precision of the paper.  ... 
arXiv:1912.03558v3 fatcat:fikxxxrginc27dx6on37ocen3u

Modular Multitask Reinforcement Learning with Policy Sketches [article]

Jacob Andreas and Dan Klein and Sergey Levine
2017 arXiv   pre-print
subgoals.  ...  Optimization is accomplished via a decoupled actor--critic training objective that facilitates learning common behaviors from multiple dissimilar reward functions.  ...  option discovery (Bacon & Precup, 2015) , and several standard policy gradient baselines.  ... 
arXiv:1611.01796v2 fatcat:mdxg3iufvrb3tarbpdvvdmcu6m

Varieties of Metacognition in Natural and Artificial Systems [chapter]

Aaron Sloman
2011 Metareasoning  
This paper, partly based on philosophical analysis of requirements for robots in complex 3-D environments, surveys varieties of meta-cognition, drawing attention to requirements that drove biological evolution  ...  Others aim to understand general principles of information-processing machines with various kinds of intelligence, whether natural or artificial, including humans and human-like systems.  ...  Recent ideas came from discussions with Jackie Chappell about non-human organisms and nature-nurture trade-offs, and members of the EU-Funded CoSy and CogX robotics projects:  ... 
doi:10.7551/mitpress/9780262014809.003.0020 fatcat:5uh2tb43afhxre5ybukjiyslm4

Diversity-Driven Extensible Hierarchical Reinforcement Learning [article]

Yuhang Song, Jianyi Wang, Thomas Lukasiewicz, Zhenghua Xu, Mai Xu
2018 arXiv   pre-print
Hierarchical reinforcement learning (HRL) has recently shown promising advances on speeding up learning, improving the exploration, and discovering intertask transferable skills.  ...  However, existing implementations of this diversity assumption usually have their own drawbacks, which makes them inapplicable to HRL with multiple levels.  ...  The diversity assumption is prevailing in the recent works of option discovery.  ... 
arXiv:1811.04324v2 fatcat:k3vdmuk5e5fu5psweautl4f7gm
« Previous Showing results 1 — 15 out of 469 results