Guided Depth Map Super-Resolution using Recumbent Y Network

Tao Li, Xiucheng Dong, Hongwei Lin
<span title="">2020</span> <i title="Institute of Electrical and Electronics Engineers (IEEE)"> <a target="_blank" rel="noopener" href="" style="color: black;">IEEE Access</a> </i> &nbsp;
Low spatial resolution is a well-known problem for depth maps captured by low-cost consumer depth cameras. Depth map super-resolution (SR) can be used to enhance the resolution and improve the quality of depth maps. In this paper, we propose a recumbent Y network (RYNet) to integrate the depth information and intensity information for depth map SR. Specifically, we introduce two weight-shared encoders to respectively learn multi-scale depth and intensity features, and a single decoder to
more &raquo; ... ly fuse depth information and intensity information for reconstruction. We also design a residual channel attention based atrous spatial pyramid pooling structure to further enrich the feature's scale diversity and exploit the correlations between multi-scale feature channels. Furthermore, the violations of co-occurrence assumption between depth discontinuities and intensity edges will generate texture-transfer and depth-bleeding artifacts. Thus, we propose a spatial attention mechanism to mitigate the artifacts by adaptively learning the spatial relevance between intensity features and depth features and reweighting the intensity features before fusion. Experimental results demonstrate the superiority of the proposed RYNet over several state-of-the-art depth map SR methods. INDEX TERMS Depth map super-resolution, convolutional neural network, UNet network, atrous spatial pyramid pooling, attention mechanism. VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see XIUCHENG DONG received the B.S. and M.S. degrees from Chongqing University, China, in 1985 and 1990, respectively. He is currently a Professor with the He is also the Dean of the Electrical Engineering and Electronic Information, Xihua University. His research interests include modern control theory, adaptive control, and robotics. HONGWEI LIN (Member, IEEE) received the M.S. degree in communication and information System from Xidian University, China, in 2011, and the Ph.D. degree in communication and information system from Sichuan University, China, in 2019. He is currently a Lecturer with the Electrical Engineering, Northwest Minzu University. His research interests include image processing and video compression and communication.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="">doi:10.1109/access.2020.3007667</a> <a target="_blank" rel="external noopener" href="">fatcat:2ncqv3rbxncwhesslnzyeylyva</a> </span>
<a target="_blank" rel="noopener" href="" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href=""> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> </button> </a>