RGB Guided Depth Map Super-Resolution with Coupled U-Net

Yingjie Cui, Qingmin Liao, Wenming Yang, Jing-Hao Xue
2021 2021 IEEE International Conference on Multimedia and Expo (ICME)  
The depth maps captured by RGB-D cameras usually are of low resolution, entailing recent efforts to develop depth super-resolution (DSR) methods. However, several problems remain in existing DSR methods. First, conventional DSR methods often suffer from unexpected artifacts. Secondly, high-resolution (HR) RGB features and low-resolution (LR) depth features are often fused in shallow layers only. Thirdly, only the last layer of features is used for reconstruction. To address the above problems,
more » ... e propose Coupled U-Net (CU-Net), a new color image guided DSR method built on two U-Net branches for HR color images and LR depth maps, respectively. The CU-Net embeds a dual skip connection structure to leverage the feature interaction of the two branches, and a multi-scale fusion to fuse the deeper and multi-scale features of two branch decoders for more effective feature reconstruction. Moreover, a channel attention module is proposed to eliminate artifacts. Extensive experiments show that the proposed CU-Net outperforms state-of-the-art methods.
doi:10.1109/icme51207.2021.9428096 fatcat:5jubanzrxzbwlkm55mftf45jki