Composing Meta-Policies for Autonomous Driving Using Hierarchical Deep Reinforcement Learning.

This paper reports results from experiments using Deep Reinforcement Learning on a continuous-state, discrete-action autonomous driving simulator. ... Rather than learning new control policies for each new task, it is possible, when tasks share some structure, to compose a "meta-policy" from previously learned policies. ... This has been studied as a variant of Hierarchical Reinforcement Learning (HRL) [12, 4, 8, 14, 26, 9] , where a Reinforcement Learning algorithm learns a state-dependent meta-policy that switches between ...

arXiv:1711.01503v1 fatcat:nk4nre7j65falaofacwiljr2ne

We first provide an overview of the tasks in autonomous driving systems, reinforcement learning algorithms and applications of DRL to AD systems. ... Deep Reinforcement Learning (DRL) has become increasingly powerful in recent years, with notable achievements such as Deepmind's AlphaGo. ... Their results show that learning a new policy for a new task takes longer than a meta-policy learnt using basis policies. ...

arXiv:1901.01536v3 fatcat:y3gck5rznjglvim4gem5dvb2ue

Multiple Versions

We construct a layered representation of control policies composed of low- level behaviors and a meta-level policy. ... The low-level behaviors enable the robot to locomote in a particular environment while avoiding obstacles, and the meta-level policy actively selects the low-level behavior most appropriate for the current ... In order to learn the meta-level policy Ω (and the low-level policies π i for i ∈ 1 . . . n in certain cases), we make use of Deep Q-learning [4] . ...

arXiv:1803.01446v3 fatcat:73lmhymgs5hetmw63twko7nb4m

Multiple Versions

These deep learning methods can yield highly effective results according to the data set size, data set quality, the methods used in feature extraction, the hyper parameter set used in deep learning models ... This is an important open point in artificial neural networks and deep learning models. ... (a) Meta-reinforcement learning (stack of sub-policies representation) [33] and (b) meta-reinforcement learning (inner-outer loop representation) [34] . ...

doi:10.5772/intechopen.92172 fatcat:sgmxtwloa5bbzb5sp7tpi75i3y

Open Access

Deep Reinforcement Learning (DRL) aims to create intelligent agents that can learn to solve complex problems efficiently in a real-world environment. ... Typically, two learning goals: adaptation and generalization are used for baselining DRL algorithm's performance on different tasks and domains. ... In recent years, Deep Reinforcement Learning (DRL) is gaining traction in the AI community due to a plethora of emerging approaches being used for developing efficient policies to solve complex problems ...

arXiv:2202.08444v1 fatcat:xc3bgq3jdngazlplw66ejtax6q

Open Access

With the development of deep representation learning, the domain of reinforcement learning (RL) has become a powerful learning framework now capable of learning complex policies in high dimensional environments ... in real world deployment of autonomous driving agents. ... Index Terms-Deep reinforcement learning, Autonomous driving, Imitation learning, Inverse reinforcement learning, Controller learning, Trajectory optimisation, Motion planning, Safe reinforcement learning ...

arXiv:2002.00444v2 fatcat:axj3ohhjwzdrxp6dgpfqvctv2i

Multiple Versions

Tactical driving decision making is crucial for autonomous driving systems and has attracted considerable interest in recent years. ... In this paper, we propose several practical components that can speed up deep reinforcement learning algorithms towards tactical decision making tasks: 1) non-uniform action skipping as a more stable alternative ... Conclusion Deep reinforcement learning is a promising framework to tackle the tactical decision making tasks in autonomous driving systems. ...

arXiv:1802.00332v1 fatcat:syk22kvwqnac3owfvaobk66z7q

Artificial intelligence has made big steps forward with reinforcement learning (RL) in the last century, and with the advent of deep learning (DL) in the 90s, especially, the breakthrough of convolutional ... The adoption of DL neural networks in RL, in the first decade of the 21 century, led to an end-toend framework allowing a great advance in human-level agents and autonomous systems, called deep reinforcement ... Many solutions have been proposed in that sense like: • Hierarchical Deep Reinforcement Learning [69] or H-DQN where meta-controller learns the optimal goal policy and provides it to the controller that ...

doi:10.14313/jamris_3-2018/15 fatcat:wn5i7y7tgfhvnhz3u5xkqlgvpe

Open Access

To address driving in near-accident scenarios, we propose a hierarchical reinforcement and imitation learning (H-ReIL) approach that consists of low-level policies learned by IL for discrete driving modes ... However, reinforcement learning (RL) and imitation learning (IL), two widely-used policy learning methods, cannot model rapid phase transitions and are not scalable to fully cover all the states. ... ACKNOWLEDGMENTS The authors thank Derek Phillips for the help with CARLA simulator, Wentao Zhong and Jiaqiao Zhang for additional experiments with H-REIL, and acknowledge funding by FLI grant RFP2-000. ...

arXiv:2007.00178v1 fatcat:doww7l34abbnvpmb5qbuo4hqty

Conventionally, model-based reinforcement learning (MBRL) aims to learn a global model for the dynamics of the environment. ... over prior hierarchical RL methods for unsupervised skill discovery. ... The performance of the meta-controller is constrained by the low-level policy, however, this hierarchical scheme is agnostic to the algorithm used to learn the low-level policy. ...

arXiv:1907.01657v2 fatcat:hm4nmah6w5csxlinn7r3oiaqqy

Multiple Versions

Then we discuss important mechanisms for RL, including attention and memory, unsupervised learning, hierarchical RL, multi-agent RL, relational RL, and learning to learn. ... We discuss deep reinforcement learning in an overview style. We draw a big picture, filled with details. ... The authors propose policy-space response oracle (PSRO), and its approximation, deep cognitive hierarchies (DCH), to compute best responses to a mixture of policies using deep RL, and to compute new meta-strategy ...

arXiv:1810.06339v1 fatcat:kp7atz5pdbeqta352e6b3nmuhy

Although deep reinforcement learning has become a promising machine learning approach for sequential decision-making problems, it is still not mature enough for high-stake domains such as autonomous driving ... In such contexts, a learned policy needs for instance to be interpretable, so that it can be inspected before any deployment (e.g., for safety and verifiability reasons). ... In this way, the meta-MDP can be solved by classic deep reinforcement learning algorithms. ...

arXiv:2112.13112v2 fatcat:wixrobyiwfabnfle2ftjfk64ki

Open Access Multiple Versions

Experiments on benchmark tasks, including a large scale autonomous driving task in the CARLA simulator, show promising results in scaling up IRL for tasks with complex reward functions. ... An inverse reinforcement learning (IRL) agent learns to act intelligently by observing expert demonstrations and learning the expert's underlying reward function. ... Large-Scale Cost Function Learning for Autonomous Driving We extend CHIRL to learn the large-scale cost function for autonomous driving in the high-fidelity CARLA simulator. ...

arXiv:2202.12597v1 fatcat:s4mj6zwhrzbefed4ymyt64xqwi

Open Access

This survey focuses on deep learning solutions that target learning control policies for robotics applications. ... We carry out our discussions on the two main paradigms for learning control with deep networks: deep reinforcement learning and imitation learning. ... of deep reinforcement learning (DRL) methods. ...

arXiv:1612.07139v4 fatcat:znbcze2jzjeshaciko7amxwxc4

Multiple Versions

In this paper we present our vision for the future of autonomous networking. ... We argue that only a holistic architecture based on hierarchies of hybrid learning, functional composition, and online experimental evaluation is expressive and capable enough to realise true autonomy ... application areas for ANNs, enabling Deep Reinforcement Learning to best humans at board (go, chess) and video games [48] . ...

doi:10.13052/jicts2245-800x.927 fatcat:dsvrtxnyy5fy5cmicfs2olmyfy

Composing Meta-Policies for Autonomous Driving Using Hierarchical Deep Reinforcement Learning [article]

Preserved Fulltext

Exploring applications of deep reinforcement learning for real-world autonomous driving systems [article]

Preserved Fulltext

Other Versions

Learning to Sequence Robot Behaviors for Visual Navigation [article]

Preserved Fulltext

Other Versions

Explainable Artificial Intelligence (xAI) Approaches and Deep Meta-Learning Models [chapter]

Preserved Fulltext

A Survey on Deep Reinforcement Learning-based Approaches for Adaptation and Generalization [article]

Preserved Fulltext

Deep Reinforcement Learning for Autonomous Driving: A Survey [article]

Preserved Fulltext

Other Versions

Elements of Effective Deep Reinforcement Learning towards Tactical Driving Decision Making [article]

Preserved Fulltext

Deep Reinforcement Learning Overview of the state of the Art

Preserved Fulltext

Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving [article]

Preserved Fulltext

Dynamics-Aware Unsupervised Discovery of Skills [article]

Preserved Fulltext

Other Versions

Deep Reinforcement Learning [article]

Preserved Fulltext

A Survey on Interpretable Reinforcement Learning [article]

Preserved Fulltext

Context-Hierarchy Inverse Reinforcement Learning [article]

Preserved Fulltext

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation [article]

Preserved Fulltext

Other Versions

Evolutionary Autonomous Networks

Preserved Fulltext