Just Look at the Image: Viewpoint-Specific Surface Normal Prediction for Improved Multi-View Reconstruction

Silvano Galliani, Konrad Schindler
2016 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
We present a multi-view reconstruction method that combines conventional multi-view stereo (MVS) with appearance-based normal prediction, to obtain dense and accurate 3D surface models. Reliable surface normals reconstructed from multi-view correspondence serve as training data for a convolutional neural network (CNN), which predicts continuous normal vectors from raw image patches. By training from known points in the same image, the prediction is specifically tailored to the materials and
more » ... ting conditions of the particular scene, as well as to the precise camera viewpoint. It is therefore a lot easier to learn than generic single-view normal estimation. The estimated normal maps, together with the known depth values from MVS, are integrated to dense depth maps, which in turn are fused into a 3D model. Experiments on the DTU dataset show that our method delivers 3D reconstructions with the same accuracy as MVS, but with significantly higher completeness.
doi:10.1109/cvpr.2016.591 dblp:conf/cvpr/GallianiS16 fatcat:6ba7locv45gx5mz6wrhlpgmo3e