Autoscanning for coupled scene reconstruction and proactive object analysis

Kai Xu, Hui Huang, Yifei Shi, Hao Li, Pinxin Long, Jianong Caichen, Wei Sun, Baoquan Chen
2015 ACM Transactions on Graphics  
a) (b) (c) Figure 1: Autonomous scene scanning and reconstruction with object analysis aided by robot pushing. (a): A PR2 robot with one arm equipped with a depth camera interacts with a cluttered table-top scene, to scan and extract the objects on top of it. (b): The reconstructed scene with extracted individual objects shown with distinct colors. (c): Zoomed-in views of the room corners. Abstract Detailed scanning of indoor scenes is tedious for humans. We propose autonomous scene scanning by
more » ... a robot to relieve humans from such a laborious task. In an autonomous setting, detailed scene acquisition is inevitably coupled with scene analysis at the required level of detail. We develop a framework for object-level scene reconstruction coupled with object-centric scene analysis. As a result, the autoscanning and reconstruction will be object-aware, guided by the object analysis. The analysis is, in turn, gradually improved with progressively increased object-wise data fidelity. In realizing such a framework, we drive the robot to execute an iterative analyze-and-validate algorithm which interleaves between object analysis and guided validations. The object analysis incorporates online learning into a robust graphcut based segmentation framework, achieving a global update of object-level segmentation based on the knowledge gained from robot-operated local validation. Based on the current analysis, the robot performs proactive validation over the scene with physical push and scan refinement, aiming at reducing the uncertainty of both object-level segmentation and object-wise reconstruction. We propose a joint entropy to measure such uncertainty based on segmentation confidence and reconstruction quality, and formulate the selection of validation actions as a maximum information gain problem. The output of our system is a reconstructed scene with both object extraction and object-wise geometry fidelity.
doi:10.1145/2816795.2818075 fatcat:cnhqk4f6jvhd3btd3zlpvv44iq