Reliability Fusion of Time-of-Flight Depth and Stereo Geometry for High Quality Depth Maps

Jiejie Zhu, Liang Wang, Ruigang Yang, J E Davis, Zhigeng Pan
2011 IEEE Transactions on Pattern Analysis and Machine Intelligence  
Time-of-flight range sensors have error characteristics, which are complementary to passive stereo. They provide real-time depth estimates in conditions where passive stereo does not work well, such as on white walls. In contrast, these sensors are noisy and often perform poorly on the textured scenes where stereo excels. We explore their complementary characteristics and introduce a method for combining the results from both methods that achieve better accuracy than either alone. In our fusion
more » ... framework, the depth probability distribution functions from each of these sensor modalities are formulated and optimized. Robust and adaptive fusion is built on a pixel-wise reliability weighting function calculated for each method. In addition, since time-of-flight devices have primarily been used as individual sensors, they are typically poorly calibrated. We introduce a method that substantially improves upon the manufacturer's calibration. We demonstrate that our proposed techniques lead to improved accuracy and robustness on an extensive set of experimental results.
doi:10.1109/tpami.2010.172 pmid:20820074 fatcat:vugntoqm6fg3jgzft3qe3afiiy