47 Hits in 6.9 sec

Hysteretic q-learning :an algorithm for decentralized reinforcement learning in cooperative multi-agent teams

Laetitia Matignon, Guillaume J. Laurent, Nadine Le Fort-Piat
2007 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems  
We report an investigation of existing algorithms for the learning of coordination in cooperative MAS, and suggest a Q-Learning extension for ILs, called Hysteretic Q-Learning.  ...  The article focuses on decentralized reinforcement learning (RL) in cooperative MAS, where a team of independent learning robots (IL) try to coordinate their individual behavior to reach a coherent joint  ...  Hysteretic Q-learning In a MAS, the reinforcement received by an agent relies on actions chosen by the team.  ... 
doi:10.1109/iros.2007.4399095 dblp:conf/iros/MatignonLF07 fatcat:dxhajv77b5e6zlnquch6wxjcba

Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems

Laetitia Matignon, Guillaume J. Laurent, Nadine Le Fort-Piat
2012 Knowledge engineering review (Print)  
Those algorithms are Q-learning variants: decentralized Q-learning, distributed Q-learning, hysteretic Q-learning, recursive FMQ and WoLF PHC.  ...  In the framework of fully cooperative multi-agent systems, independent (non-communicative) agents that learn by reinforcement must overcome several difficulties to manage to coordinate.  ...  Conclusion This paper presented a comprehensive review of reinforcement learning algorithms for independent agents in cooperative multi-agent systems.  ... 
doi:10.1017/s0269888912000057 fatcat:j2unyb75c5a3lmpvdny3ex77ei

Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability [article]

Shayegan Omidshafiei, Jason Pazis, Christopher Amato, Jonathan P. How, John Vian
2017 arXiv   pre-print
This paper formalizes and addresses the problem of multi-task multi-agent reinforcement learning under partial observability.  ...  Approaches that learn specialized policies for individual tasks face problems when applied to the real world: not only do agents have to learn and store distinct policies for each task, but in practice  ...  Acknowledgements The authors thank the anonymous reviewers for their insightful feedback and suggestions.  ... 
arXiv:1703.06182v4 fatcat:pt76xj24snafziyymv4nnqkqsy

Lenient Multi-Agent Deep Reinforcement Learning [article]

Gregory Palmer, Karl Tuyls, Daan Bloembergen, Rahul Savani
2018 arXiv   pre-print
This introduces optimism in the value-function update, and has been shown to facilitate cooperation in tabular fully-cooperative multi-agent reinforcement learning problems.  ...  However, care is required when using ERMs for multi-agent deep reinforcement learning (MA-DRL), as stored transitions can become outdated because agents update their policies in parallel [11].  ...  Hysteretic Q-learning is a form of optimistic learning with a strong empirical track record in fully-observable multi-agent reinforcement learning [3, 20, 37] .  ... 
arXiv:1707.04402v2 fatcat:ozvqkfba7jf7vhpj7jg6imtsci

Adaptive learning: A new decentralized reinforcement learning approach for cooperative multiagent systems

Li Meng-Lin, Chen Shao-Fei, Chen Jing
2020 IEEE Access  
By studying the paradigm of centralized training and decentralized execution(CTDE), a multi-agent reinforcement learning algorithm for implicit coordination based on TD error is proposed.  ...  Moreover, the variance of the training results is more stable than that of the hysteretic Q learning(HQL) algorithm.  ...  This laid the foundation for the use of the deep Q reinforcement learning (RL) algorithm in a multi-agent environment.  ... 
doi:10.1109/access.2020.2997899 fatcat:nyhq2q6u5jhrpknp634xro3wja

Multi-Agent Reinforcement Learning: A Review of Challenges and Applications

Lorenzo Canese, Gian Carlo Cardarilli, Luca Di Di Nunzio, Rocco Fazzolari, Daniele Giardino, Marco Re, Sergio Spanò
2021 Applied Sciences  
In this review, we present an analysis of the most used multi-agent reinforcement learning algorithms.  ...  The described multi-agent algorithms are compared in terms of the most important characteristics for multi-agent reinforcement learning applications—namely, nonstationarity, scalability, and observability  ...  In the context of cooperative games, hysteretic Q-learning [26] is an algorithm that improves the performance of standard independent learners approaches.  ... 
doi:10.3390/app11114948 fatcat:fqggdltjt5hrjojpjm5ipmapnm

Learning Multi-Robot Decentralized Macro-Action-Based Policies via a Centralized Q-Net [article]

Yuchen Xiao, Joshua Hoffman, Tian Xia, Christopher Amato
2020 arXiv   pre-print
Decentralized multi-agent reinforcement learning methods have difficulty learning decentralized policies because of the environment appearing to be non-stationary due to other agents also learning at the  ...  In this paper, we address this challenge by proposing a macro-action-based decentralized multi-agent double deep recurrent Q-net (MacDec-MADDRQN) which trains each decentralized Q-net using a centralized  ...  Learning Macro-Action-Based Deep Q-Nets Although there has been several popular multi-agent deep reinforcement learning methods achieving impressive performance in cooperative as well as competitive domains  ... 
arXiv:1909.08776v2 fatcat:dbtujbhlbjcq5gqullsxq6hfzq

Likelihood Quantile Networks for Coordinating Multi-Agent Reinforcement Learning [article]

Xueguang Lyu, Christopher Amato
2020 arXiv   pre-print
When multiple agents learn in a decentralized manner, the environment appears non-stationary from the perspective of an individual agent due to the exploration and learning of the other agents.  ...  Recently proposed deep multi-agent reinforcement learning methods have tried to mitigate this non-stationarity by attempting to determine which samples are from other agent exploration or suboptimality  ...  CONCLUSION This paper describes a novel distributional RL method for improving performance in cooperative multi-agent reinforcement learning settings.  ... 
arXiv:1812.06319v6 fatcat:6lsfmhoww5ffjlkwvuwrw4z5je

Accelerated Method Based on Reinforcement Learning and Case Base Reasoning in Multi agent Systems

Sara Esfandiari, Behrooz Masoumi, Mohammad Reza Meybodi, Abdolkarim Niazi
2012 International Journal of Computer Applications  
General Terms Multi Agent Learning , Machine Learning .  ...  Q-learning.  ...  In a fully cooperative MG (or team MG) called a multi-agent MDP (or MMDP), all agents share the same reward function.  ... 
doi:10.5120/4597-6796 fatcat:dapry44oivfu3m3kxfr277o7fm

Logical Team Q-learning: An approach towards factored policies in cooperative MARL [article]

Lucas Cassano, Ali H. Sayed
2021 arXiv   pre-print
We address the challenge of learning factored policies in cooperative MARL scenarios. In particular, we consider the situation in which a team of agents collaborates to optimize a common cost.  ...  The main contribution of this work is the introduction of Logical Team Q-learning (LTQL).  ...  Hysteretic Q-learning: an algorithm for decentralized reinforcement learning in cooperative multi-agent teams. In Proc.  ... 
arXiv:2006.03553v2 fatcat:6zkiz4rjszac5enpwb43vqgjua

Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications [article]

Thanh Thi Nguyen, Ngoc Duy Nguyen, Saeid Nahavandi
2019 arXiv   pre-print
Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems.  ...  schemes, multi-agent transfer learning.  ...  Q-learning algorithm proposed in [111] .  ... 
arXiv:1812.11794v2 fatcat:dkmnfhdsrrepzd277nhzouymzq

Decentralized Multi-Agents by Imitation of a Centralized Controller [article]

Alex Tong Lin, Mark J. Debord, Katia Estabridis, Gary Hewer, Guido Montufar, Stanley Osher
2021 arXiv   pre-print
This is in contrast to other multi-agent learning algorithms that, for example, can require more specific structures.  ...  This framework has the flexibility to use any reinforcement learning algorithm to obtain the expert as well as any imitation learning algorithm to obtain the decentralized agents.  ...  Introduction Reinforcement Learning (RL) is the problem of finding an action policy that maximizes reward for an agent embedded in an environment (Sutton and Barto, 2018) .  ... 
arXiv:1902.02311v4 fatcat:3snvbulww5fcrn5dhafwhmgkjq

Designing Decentralized Controllers for Distributed-Air-Jet MEMS-Based Micromanipulators by Reinforcement Learning

Laëtitia Matignon, Guillaume J. Laurent, Nadine Le Fort-Piat, Yves-André Chapuis
2010 Journal of Intelligent and Robotic Systems  
In this paper, we investigate reinforcement learning control approaches in order to position and convey an object.  ...  We show how to apply reinforcement learning in a decentralized perspective and in order to address the global-local trade-off.  ...  Acknowledgements The authors gratefully acknowledge Joël Agnus and David Guibert from the FEMTO-ST Institute for their technical assistance.  ... 
doi:10.1007/s10846-010-9396-9 fatcat:ucqcciu3yfhjvnonar7mpw2bcy

Decentralized Multi-Agent Control of a Manipulator in Continuous Task Learning

Asad Ali Shahid, Jorge Said Vidal Sesin, Damjan Pecioski, Francesco Braghin, Dario Piga, Loris Roveda
2021 Applied Sciences  
Multiple variations of the multi-agent framework have been proposed and tested in this research, comparing the achieved performance w.r.t. a centralized (i.e., single-agent) control action learning framework  ...  Within this context, this paper focuses on the decentralization of the robot control action learning and (re)execution considering a generic multi-DoF manipulator.  ...  Additionally, the Hysteretic Q-Learning algorithm [23] can be used for decentralized reinforcement learning.  ... 
doi:10.3390/app112110227 fatcat:uy2jqpq4rfearbcrvlos2syngm

Dynamic correlation matrix based multi-Q learning for a multi-robot system

Hongliang Guo, Yan Meng
2008 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems  
In this paper, we propose a dynamic correlation matrix based multi-Q learning (DCM-MultiQ) method for a distributed multi-robot system.  ...  Multi-robot reinforcement learning is a very challenging area due to several issues, such as large state spaces, difficulty in reward assignment, nondeterministic action selections, and difficulty in merging  ...  Hysteretic Q-learning method proposed in [11] has improved traditional Q-learning through varying learning rates, which aims at forcing agents to converge to an optimal policy.  ... 
doi:10.1109/iros.2008.4651021 dblp:conf/iros/GuoM08 fatcat:xnpcyqvwrngylkwqzlqjaxj6py
« Previous Showing results 1 — 15 out of 47 results