ODE-CNN: Omnidirectional Depth Extension Networks [article]

Xinjing Cheng, Peng Wang, Yanqi Zhou, Chenye Guan, Ruigang Yang
<span title="2020-07-03">2020</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Omnidirectional 360 camera proliferates rapidly for autonomous robots since it significantly enhances the perception ability by widening the field of view(FoV). However, corresponding 360 depth sensors, which are also critical for the perception system, are still difficult or expensive to have. In this paper, we propose a low-cost 3D sensing system that combines an omnidirectional camera with a calibrated projective depth camera, where the depth from the limited FoV can be automatically
more &raquo; ... to the rest of the recorded omnidirectional image. To accurately recover the missing depths, we design an omnidirectional depth extension convolutional neural network(ODE-CNN), in which a spherical feature transform layer(SFTL) is embedded at the end of feature encoding layers, and a deformable convolutional spatial propagation network(D-CSPN) is appended at the end of feature decoding layers. The former resamples the neighborhood of each pixel in the omnidirectional coordination to the projective coordination, which reduces the difficulty of feature learning, and the later automatically finds a proper context to well align the structures in the estimated depths via CNN w.r.t. the reference image, which significantly improves the visual quality. Finally, we demonstrate the effectiveness of proposed ODE-CNN over the popular 360D dataset and show that ODE-CNN significantly outperforms (relatively 33 in-depth error) other state-of-the-art (SoTA) methods.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2007.01475v1">arXiv:2007.01475v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/sz6eicbf6vbbnl6z6stockshdm">fatcat:sz6eicbf6vbbnl6z6stockshdm</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200710052144/https://arxiv.org/pdf/2007.01475v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2007.01475v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>