Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework [article]

Wanxin Jin, Zhaoran Wang, Zhuoran Yang, Shaoshuai Mou
<span title="2021-01-12">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
This paper develops a Pontryagin Differentiable Programming (PDP) methodology, which establishes a unified framework to solve a broad class of learning and control tasks. The PDP distinguishes from existing methods by two novel techniques: first, we differentiate through Pontryagin's Maximum Principle, and this allows to obtain the analytical derivative of a trajectory with respect to tunable parameters within an optimal control system, enabling end-to-end learning of dynamics, policies, or/and
more &raquo; ... control objective functions; and second, we propose an auxiliary control system in the backward pass of the PDP framework, and the output of this auxiliary control system is the analytical derivative of the original system's trajectory with respect to the parameters, which can be iteratively solved using standard control tools. We investigate three learning modes of the PDP: inverse reinforcement learning, system identification, and control/planning. We demonstrate the capability of the PDP in each learning mode on different high-dimensional systems, including multi-link robot arm, 6-DoF maneuvering quadrotor, and 6-DoF rocket powered landing.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1912.12970v5">arXiv:1912.12970v5</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/7sse7c3bvzf7tjyu2smubzh3ri">fatcat:7sse7c3bvzf7tjyu2smubzh3ri</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210116030751/https://arxiv.org/pdf/1912.12970v5.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/bc/7d/bc7d2215a2fde297d3284ea55a20ba0b3249355b.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1912.12970v5" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>