HighRes-MVSNet: A Fast Multi-View Stereo Network for Dense 3D Reconstruction from High-Resolution Images

Rafael Weilharter, Friedrich Fraundorfer
2021 IEEE Access  
We propose an end-to-end deep learning architecture for 3D reconstruction from high-resolution images. While many approaches focus on improving reconstruction quality alone, we primarily focus on decreasing memory requirements in order to exploit the abundant information provided by modern high-resolution cameras. Towards this end, we present HighRes-MVSNet, a convolutional neural network with a pyramid encoder-decoder structure searching for depth correspondences incrementally over a
more » ... fine hierarchy. The first stage of our network encodes the image features to a much smaller resolution in order to significantly reduce the memory requirements. Additionally, we limit the depth search range in every hierarchy level to the vicinity of the previous prediction. In this manner, we are able to produce highly accurate 3D models while only using a fraction of the GPU memory and runtime of previous methods. Although our method is aimed at much higher resolution images, we are still able to produce state-of-the-art results on the Tanks and Temples benchmark and achieve outstanding scores on the DTU benchmark. INDEX TERMS Convolutional neural network, dense 3D reconstruction, multi-view stereo.
doi:10.1109/access.2021.3050556 fatcat:ssupu7viubavdmij4fdgusenzq