Real-Time Semantic Segmentation-Based Stereo Reconstruction

Vlad-Cristian Miclea, Sergiu Nedevschi
2019 IEEE transactions on intelligent transportation systems (Print)  
In this paper, we propose a novel semantic segmentation-based stereo reconstruction method that can keep up with the accuracy of the state-of-the art approaches while running in real time. The solution follows the classic stereo pipeline, each step in the stereo workflow being enhanced by additional information from semantic segmentation. Therefore, we introduce several improvements to computation, aggregation, and optimization by adapting existing techniques to integrate additional surface
more » ... rmation given by each semantic class. For the cost computation and optimization steps, we propose new genetic algorithms that can incrementally adjust the parameters for better solutions. Furthermore, we propose a new postprocessing edge-aware filtering technique relying on an improved convolutional neural network (CNN) architecture for disparity refinement. We obtain the competitive results at 30 frames/s, including segmentation.
doi:10.1109/tits.2019.2913883 fatcat:jl2t5lit2nfmnfilv6lrbwz4pi