Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation [article]

Yu Xiang, Christopher Xie, Arsalan Mousavian, Dieter Fox
2021 arXiv   pre-print
Segmenting unseen objects in cluttered scenes is an important skill that robots need to acquire in order to perform tasks in new environments. In this work, we propose a new method for unseen object instance segmentation by learning RGB-D feature embeddings from synthetic data. A metric learning loss function is utilized to learn to produce pixel-wise feature embeddings such that pixels from the same object are close to each other and pixels from different objects are separated in the embedding
more » ... space. With the learned feature embeddings, a mean shift clustering algorithm can be applied to discover and segment unseen objects. We further improve the segmentation accuracy with a new two-stage clustering algorithm. Our method demonstrates that non-photorealistic synthetic RGB and depth images can be used to learn feature embeddings that transfer well to real-world images for unseen object instance segmentation.
arXiv:2007.15157v3 fatcat:oswra62xdngfzcckhdmqzlzrp4