Multi-objective reinforcement learning for AUV thruster failure recovery

Seyed Reza Ahmadzadeh, Petar Kormushev, Darwin G. Caldwell
2014 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL)  
This paper investigates learning approaches for discovering fault-tolerant control policies to overcome thruster failures in Autonomous Underwater Vehicles (AUV). The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the vehicle. When a fault is detected and isolated the model of the AUV is reconfigured according to the new condition. To discover a set of optimal solutions a multi-objective reinforcement learning approach is employed which can
more » ... employed which can deal with multiple conflicting objectives. Each optimal solution can be used to generate a trajectory that is able to navigate the AUV towards a specified target while satisfying multiple objectives. The discovered policies are executed on the robot in a closed-loop using AUV's state feedback. Unlike most existing methods which disregard the faulty thruster, our approach can also deal with partially broken thrusters to increase the persistent autonomy of the AUV. In addition, the proposed approach is applicable when the AUV either becomes underactuated or remains redundant in the presence of a fault. We validate the proposed approach on the model of the Girona500 AUV.
doi:10.1109/adprl.2014.7010621 dblp:conf/adprl/AhmadzadehKC14 fatcat:mqpv6rud6jan7dxboqkxm6cb3e