2,325 Hits in 3.3 sec

Risk Averse Robust Adversarial Reinforcement Learning [article]

Xinlei Pan, Daniel Seita, Yang Gao, John Canny
2019 arXiv   pre-print
In this paper we introduce risk-averse robust adversarial reinforcement learning (RARARL), using a risk-averse protagonist and a risk-seeking adversary.  ...  Recently, robust adversarial reinforcement learning (RARL) was developed, which allows efficient applications of random and systematic perturbations by a trained adversary.  ...  RISK AVERSE ROBUST ADVERSARIAL RL In this section, we formalize our risk averse robust adversarial reinforcement learning (RARARL) framework. A.  ... 
arXiv:1904.00511v1 fatcat:5jwf2nreefe7bjgpli7leupazi

Robust Risk-Sensitive Reinforcement Learning Agents for Trading Markets [article]

Yue Gao and Kry Yik Chau Lui and Pablo Hernandez-Leal
2021 arXiv   pre-print
To tackle these type of scenarios agents need to exhibit certain characteristics such as risk-awareness, robustness to perturbations and low learning variance.  ...  First, we contribute with two algorithms that use risk-averse objective functions and variance reduction techniques.  ...  Robust Risk-Sensitive Reinforcement Learning Agents for Trading Markets Algorithm 7 Risk-Averse Adversarial Averaged Q-Learning (RA3-Q) Input : Training steps T ; Exploration rate ; Number of models k;  ... 
arXiv:2107.08083v1 fatcat:owoy6a32ava3xb3f52pndgc6ay

Robust Market Making via Adversarial Reinforcement Learning [article]

Thomas Spooner, Rahul Savani
2020 arXiv   pre-print
We show that adversarial reinforcement learning (ARL) can be used to produce market marking agents that are robust to adversarial and adaptively-chosen market conditions.  ...  We empirically compare two conventional single-agent RL agents with ARL, and show that our ARL approach leads to: 1) the emergence of risk-averse behaviour without constraints or domain-specific penalties  ...  To the best of our knowledge, we are the first to apply ARL to derive trading strategies that are robust to epistemic risk. Risk-sensitive reinforcement learning.  ... 
arXiv:2003.01820v2 fatcat:vr4667xoz5ghbmpb5t7teue4w4

Relation-Aware Transformer for Portfolio Policy Learning

Ke Xu, Yifan Zhang, Deheng Ye, Peilin Zhao, Mingkui Tan
2020 Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence  
In this paper, under a deep reinforcement learning paradigm for portfolio selection, we propose a novel Relation-aware Transformer (RAT) to handle these aspects.  ...  To the best of our knowledge, we are the first to apply ARL to derive trading strategies that are robust to epistemic risk. Risk-sensitive reinforcement learning.  ...  Our adaption is useful to train robust MMs, and evaluate their performance in the presence of epistemic risk (Sections 2 and 3). • We propose an algorithm for adversarial reinforcement learning in the  ... 
doi:10.24963/ijcai.2020/633 dblp:conf/ijcai/SpoonerS20 fatcat:jrddq25inzedvl237x72nguihm

Robust Reinforcement Learning: A Review of Foundations and Recent Advances

Janosch Moos, Kay Hansel, Hany Abdulsamad, Svenja Stark, Debora Clever, Jan Peters
2022 Machine Learning and Knowledge Extraction  
Reinforcement learning (RL) has become a highly successful framework for learning in Markov decision processes (MDP).  ...  The resulting survey covers all fundamental concepts underlying the approaches to robust reinforcement learning and their recent advances.  ...  [56] present the risk-averse robust adversarial reinforcement learning algorithm (RARARL) assuming a two-player zero-sum sequential game, where the protagonist and adversary take turns in controlling  ... 
doi:10.3390/make4010013 fatcat:ifa3z7cx7rc7homa4flywxvhvi

Risk-Averse Bayes-Adaptive Reinforcement Learning [article]

Marc Rigter, Bruno Lacerda, Nick Hawes
2021 arXiv   pre-print
In this work, we address risk-averse Bayes-adaptive reinforcement learning.  ...  We show that a policy optimising CVaR in this setting is risk-averse to both the parametric uncertainty due to the prior distribution over MDPs, and the internal uncertainty due to the inherent stochasticity  ...  A related strand of work is robust adversarial reinforcement learning (Pinto et al., 2017; Pan et al., 2019) (RARL).  ... 
arXiv:2102.05762v2 fatcat:xftlgolminb2zerog67cchovka

Model-Free Risk-Sensitive Reinforcement Learning [article]

Grégoire Delétang, Jordi Grau-Moya, Markus Kunesch, Tim Genewein, Rob Brekelmans, Shane Legg, Pedro A. Ortega
2021 arXiv   pre-print
We extend temporal-difference (TD) learning in order to obtain risk-sensitive, model-free reinforcement learning algorithms.  ...  Since the Gaussian free energy is known to be a certainty-equivalent sensitive to the mean and the variance, the learning rule has applications in risk-sensitive decision-making.  ...  Reinforcement learning. Next we applied the risk-sensitive learning rule to RL in a simple gridworld.  ... 
arXiv:2111.02907v1 fatcat:sw3d2mske5hhtkwbln5wz25xj4

Bayesian Robust Optimization for Imitation Learning [article]

Daniel S. Brown, Scott Niekum, Marek Petrik
2020 arXiv   pre-print
Existing safe imitation learning approaches based on IRL deal with this uncertainty using a maxmin framework that optimizes a policy under the assumption of an adversarial reward function, whereas risk-neutral  ...  To provide a bridge between these two extremes, we propose Bayesian Robust Optimization for Imitation Learning (BROIL).  ...  Instead of assuming either a purely adversarial environment or a risk-neutral one, we propose the first inverse reinforcement learning algorithm capable of appropriately balancing caution with expected  ... 
arXiv:2007.12315v3 fatcat:a3yro3z2ffhxzd7rsk5nzrfhla

Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning [article]

Inaam Ilahi, Muhammad Usama, Junaid Qadir, Muhammad Umar Janjua, Ala Al-Fuqaha, Dinh Thai Hoang, Dusit Niyato
2021 arXiv   pre-print
Deep Reinforcement Learning (DRL) has numerous applications in the real world thanks to its outstanding ability in quickly adapting to the surrounding environments.  ...  We first cover some fundamental backgrounds about DRL and present emerging adversarial attacks on machine learning techniques.  ...  They propose riskaverse robust adversarial reinforcement learning (RARARL), using a risk-averse agent and a risk-seeking adversary.  ... 
arXiv:2001.09684v2 fatcat:sxl2t2wd5jf2bmk3qxnlrtmrvu

ACReL: Adversarial Conditional value-at-risk Reinforcement Learning [article]

M. Godbout, M. Heuillet, S. Chandra, R. Bhati, A. Durand
2022 arXiv   pre-print
This can be addressed by using the Conditional-Value-at-Risk (CVaR) objective to instill risk-aversion in learned policies.  ...  In this paper, we propose Adversarial Cvar Reinforcement Learning (ACReL), a novel adversarial meta-algorithm to optimize the CVaR objective in RL.  ...  The closest work to ours would be Robust Adversarial Reinforcement Learning (RARL) [Pinto et al., 2017] , where a policy is learned by allowing an adversary to apply external forces to the environment  ... 
arXiv:2109.09470v2 fatcat:addas4uwhvag3gm5jnne4nfcby

SAAC: Safe Reinforcement Learning as an Adversarial Game of Actor-Critics [article]

Yannis Flet-Berliac, Debabrota Basu
2022 arXiv   pre-print
Although Reinforcement Learning (RL) is effective for sequential decision-making problems under uncertainty, it still fails to thrive in real-world systems where risk or safety is a binding constraint.  ...  Finally, for challenging continuous control tasks, we demonstrate that SAAC achieves faster convergence, better efficiency, and fewer failures to satisfy the safety constraints than risk-averse distributional  ...  Introduction Reinforcement Learning (RL) is a paradigm of Machine Learning (ML) that addresses the problem of sequential decision making and learning under incomplete information (Puterman, 2014; Sutton  ... 
arXiv:2204.09424v1 fatcat:6hsjm7bn5zfixa5pnni776ylr4

Adversarial recovery of agent rewards from latent spaces of the limit order book [article]

Jacobo Roa-Vicens, Yuanbo Wang, Virgile Mison, Yarin Gal, Ricardo Silva
2019 arXiv   pre-print
Inverse reinforcement learning has proved its ability to explain state-action trajectories of expert agents by recovering their underlying reward functions in increasingly challenging environments.  ...  Recent advances in adversarial learning have allowed extending inverse RL to applications with non-stationary environment dynamics unknown to the agents, arbitrary structures of reward functions and improved  ...  Adversarial inverse reinforcement learning (AIRL) [8] extends inverse RL further, achieving the recovery of rewards robust to variations in the dynamics of the environment, while learning at the same  ... 
arXiv:1912.04242v1 fatcat:zfbo3bhxc5akzjvdgisb2ib3f4

Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [article]

Jesse Zhang, Brian Cheung, Chelsea Finn, Sergey Levine, Dinesh Jayaraman
2020 arXiv   pre-print
Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous, imperiling the RL agent, other agents, and the environment.  ...  We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk, which in turn enables relative safety through risk-averse  ...  Related Work Cautious or risk-averse learning has close connections to learning robust control policies, as well as the uncertainty estimation derived from Bayesian reinforcement learning (Ghavamzadeh  ... 
arXiv:2008.06622v1 fatcat:obkgm3b5trfzrdbtyh4rjasc2a

Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control [article]

Yuheng Wang, Margaret P. Chapman
2021 arXiv   pre-print
The first part of the review focuses on model-based risk-averse methods.  ...  We categorize and present state-of-the-art approaches, and we describe connections between such approaches and ideas from the fields of decision theory, operations research, reinforcement learning, and  ...  Kisiala [41] for offering intuitive introductions to CVaR, VaR, and coherent risk functionals, and we thank S. Coogan, M. Arcak, and C. Belta [61] for an informative summary about temporal logic.  ... 
arXiv:2109.08947v1 fatcat:wtboizdlm5ffrkc72bit6of3nu

Distributionally Robust Reinforcement Learning [article]

Elena Smirnova, Elvis Dohmatob, Jérémie Mary
2019 arXiv   pre-print
In this work, we consider risk-averse exploration in approximate RL setting.  ...  To ensure safety during learning, we propose the distributionally robust policy iteration scheme that provides lower bound guarantee on state-values.  ...  robust reinforcement learning objective (DR-RL).  ... 
arXiv:1902.08708v2 fatcat:gp2cb3hzrzdc7eyi6uj54dccga
« Previous Showing results 1 — 15 out of 2,325 results