Learning Cooperative Dynamic Manipulation Skills from Human Demonstration Videos [article]

Francesco Iodice, Yuqiang Wu, Wansoo Kim, Fei Zhao, Elena De Momi, Arash Ajoudani
<span title="2022-04-08">2022</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
This article proposes a method for learning and robotic replication of dynamic collaborative tasks from offline videos. The objective is to extend the concept of learning from demonstration (LfD) to dynamic scenarios, benefiting from widely available or easily producible offline videos. To achieve this goal, we decode important dynamic information, such as the Configuration Dependent Stiffness (CDS), which reveals the contribution of arm pose to the arm endpoint stiffness, from a
more &raquo; ... al human skeleton model. Next, through encoding of the CDS via Gaussian Mixture Model (GMM) and decoding via Gaussian Mixture Regression (GMR), the robot's Cartesian impedance profile is estimated and replicated. We demonstrate the proposed method in a collaborative sawing task with leader-follower structure, considering environmental constraints and dynamic uncertainties. The experimental setup includes two Panda robots, which replicate the leader-follower roles and the impedance profiles extracted from a two-persons sawing video.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2204.04003v1">arXiv:2204.04003v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/2fdlyus6zfgvtftuf3kouoowfy">fatcat:2fdlyus6zfgvtftuf3kouoowfy</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20220516094714/https://arxiv.org/pdf/2204.04003v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/96/23/96236a8f285844219ba4c63f3dc5c390005c6b86.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2204.04003v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>