Confidence-based progress-driven self-generated goals for skill acquisition in developmental robots

Hung Ngo, Matthew Luciw, Alexander Förster, Jürgen Schmidhuber
<span title="">2013</span> <i title="Frontiers Media SA"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/5r5ojcju2repjbmmjeu5oyawti" style="color: black;">Frontiers in Psychology</a> </i> &nbsp;
A reinforcement learning agent that autonomously explores its environment can utilize a curiosity drive to enable continual learning of skills, in the absence of any external rewards. We formulate curiosity-driven exploration, and eventual skill acquisition, as a selective sampling problem. Each environment setting provides the agent with a stream of instances. An instance is a sensory observation that, when queried, causes an outcome that the agent is trying to predict. After an instance is
more &raquo; ... erved, a query condition, derived herein, tells whether its outcome is statistically known or unknown to the agent, based on the confidence interval of an online linear classifier. Upon encountering the first unknown instance, the agent "queries" the environment to observe the outcome, which is expected to improve its confidence in the corresponding predictor. If the environment is in a setting where all instances are known, the agent generates a plan of actions to reach a new setting, where an unknown instance is likely to be encountered. The desired setting is a self-generated goal, and the plan of action, essentially a program to solve a problem, is a skill. The success of the plan depends on the quality of the agent's predictors, which are improved as mentioned above. For validation, this method is applied to both a simulated and real Katana robot arm in its "blocks-world" environment. Results show that the proposed method generates sample-efficient curious exploration behavior, which exhibits developmental stages, continual learning, and skill acquisition, in an intrinsically-motivated playful agent. Citation: Ngo H, Luciw M, Förster A and Schmidhuber J (2013) Confidence-based progress-driven self-generated goals for skill acquisition in developmental robots. Front.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3389/fpsyg.2013.00833">doi:10.3389/fpsyg.2013.00833</a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pubmed/24324448">pmid:24324448</a> <a target="_blank" rel="external noopener" href="https://pubmed.ncbi.nlm.nih.gov/PMC3840616/">pmcid:PMC3840616</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/f7amrbrrubd5jctkdjlmvayyw4">fatcat:f7amrbrrubd5jctkdjlmvayyw4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170829090606/https://fjfsdata01prod.blob.core.windows.net/articles/files/62546/pubmed-zip/.versions/1/.package-entries/fpsyg-04-00833/fpsyg-04-00833.pdf?sv=2015-12-11&amp;sr=b&amp;sig=NfFK4ZqjKW5MKH39EIm7%2Be6W%2BAYRUmjfGpQnHYudvaU%3D&amp;se=2017-08-29T09%3A06%3A16Z&amp;sp=r&amp;rscd=attachment%3B%20filename%2A%3DUTF-8%27%27fpsyg-04-00833.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/f2/c9/f2c9d52426221ca2ebdfa166c6974c5af8d13169.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.3389/fpsyg.2013.00833"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> frontiersin.org </button> </a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3840616" title="pubmed link"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> pubmed.gov </button> </a>