View-Volume Network for Semantic Scene Completion from a Single Depth Image

Yuxiao Guo, Xin Tong
2018 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence  
We introduce a View-Volume convolutional neural network (VVNet) for inferring the occupancy and semantic labels of a volumetric 3D scene from a single depth image. Our method extracts the detailed geometric features from the input depth image with a 2D view CNN and then projects the features into a 3D volume according to the input depth map via a projection layer. After that, we learn the 3D context information of the scene with a 3D volume CNN for computing the result volumetric occupancy and
more » ... emantic labels. With combined 2D and 3D representations, the VVNet efficiently reduces the computational cost, enables feature extraction from multi-channel high resolution inputs, and thus significantly improve the result accuracy. We validate our method and demonstrate its efficiency and effectiveness on both synthetic SUNCG and real NYU dataset.
doi:10.24963/ijcai.2018/101 dblp:conf/ijcai/GuoT18 fatcat:nqg6yb5bpfef7aquu3q4tpgljq