Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation

Qin Wang, Dengxin Dai, Lukas Hoyer, Luc Van Gool, Olga Fink
2021 2021 IEEE/CVF International Conference on Computer Vision (ICCV)  
Domain adaptation for semantic segmentation aims to improve the model performance in the presence of a distribution shift between source and target domain. Leveraging the supervision from auxiliary tasks (such as depth estimation) has the potential to heal this shift because many visual tasks are closely related to each other. However, such a supervision is not always available. In this work, we leverage the guidance from self-supervised depth estimation, which is available on both domains, to
more » ... ridge the domain gap. On the one hand, we propose to explicitly learn the task feature correlation to strengthen the target semantic predictions with the help of target depth estimation. On the other hand, we use the depth prediction discrepancy from source and target depth decoders to approximate the pixel-wise adaptation difficulty. The adaptation difficulty, inferred from depth, is then used to refine the target semantic segmentation pseudo-labels. The proposed method can be easily implemented into existing segmentation frameworks. We demonstrate the effectiveness of our approach on the benchmark tasks SYNTHIA-to-Cityscapes and GTA-to-Cityscapes, on which we achieve the new stateof-the-art performance of 55.0% and 56.6%, respectively. Our code is available at
doi:10.1109/iccv48922.2021.00840 fatcat:2qpvrotukzfyxg3mt4unzmhufy