Capturing Deformations of Interacting Non-rigid Objects Using RGB-D Data

Antoine Petit, Stephane Cotin, Vincenzo Lippiello, Bruno Siciliano
2018 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)  
This paper presents a method for tracking multiple interacting deformable objects undergoing rigid motions, elastic deformations and contacts, using image and point cloud data provided by an RGB-D sensor. A joint registration framework is proposed, based on physical Finite Element Method (FEM) elastic and interaction models. It first relies on a visual segmentation of the considered objects in the RGB images. The different segmented point clouds are then processed to estimate rigid
more » ... ns with on an ICP algorithm, and to determine geometrical point-to-point correspondences with the meshes. External forces resulting from these correspondences and between the current and the rigidly transformed mesh can then be derived. It provides both non-rigid and rigid data cues. A classical collision detection and response model is also integrated, giving contact forces between the objects. The deformations of the objects are estimated by solving a dynamic system balancing these external and contact forces with the internal or regularization forces computed through the FEM elastic model. This approach has been here tested on different scenarios involving two or three interacting deformable objects of various shapes, with promising results.
doi:10.1109/iros.2018.8593756 dblp:conf/iros/0003CLS18 fatcat:g5lnb7havvhvna2pdv7flmov5i