LiDAR-Based Real-Time Panoptic Segmentation via Spatiotemporal Sequential Data Fusion

Weiqi Wang, Xiong You, Jian Yang, Mingzhan Su, Lantian Zhang, Zhenkai Yang, Yingcai Kuang
2022 Remote Sensing  
Fast and accurate semantic scene understanding is essential for mobile robots to operate in complex environments. An emerging research topic, panoptic segmentation, serves such a purpose by performing the tasks of semantic segmentation and instance segmentation in a unified framework. To improve the performance of LiDAR-based real-time panoptic segmentation, this study proposes a spatiotemporal sequential data fusion strategy that fused points in "thing classes" based on accurate data
more » ... . The data fusion strategy could increase the proportion of valuable data in unbalanced datasets, and thus managed to mitigate the adverse impact of class imbalance in the limited training data. Subsequently, by improving the codec network, the multiscale features shared by semantic and instance branches were efficiently aggregated to achieve accurate panoptic segmentation for each LiDAR scan. Experiments on the publicly available dataset SemanticKITTI showed that our approach could achieve an effective balance between accuracy and efficiency, and it was also applicable to other point cloud segmentation tasks.
doi:10.3390/rs14081775 fatcat:qe4mhwschjej3gib5vkgdc5kdy