Multimodal Parameter-exploring Policy Gradients

Frank Sehnke, Alex Graves, Christian Osendorfer, Jurgen Schmidhuber
<span title="">2010</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="" style="color: black;">2010 Ninth International Conference on Machine Learning and Applications</a> </i> &nbsp;
Policy Gradients with Parameter-based Exploration (PGPE) is a novel model-free reinforcement learning method that alleviates the problem of high-variance gradient estimates encountered in normal policy gradient methods. It has been shown to drastically speed up convergence for several large-scale reinforcement learning tasks. However the independent normal distributions used by PGPE to search through parameter space are inadequate for some problems with multimodal reward surfaces. This paper
more &raquo; ... ends the basic PGPE algorithm to use multimodal mixture distributions for each parameter, while remaining efficient. Experimental results on the Rastrigin function and the inverted pendulum benchmark demonstrate the advantages of this modification, with faster convergence to better optima.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="">doi:10.1109/icmla.2010.24</a> <a target="_blank" rel="external noopener" href="">dblp:conf/icmla/SehnkeGOS10</a> <a target="_blank" rel="external noopener" href="">fatcat:6cbpsaxav5fsbasrxo6nlywafa</a> </span>
<a target="_blank" rel="noopener" href="" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href=""> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> </button> </a>