Automatic 3D Facial Expression Analysis in Videos [chapter]

Ya Chang, Marcelo Vieira, Matthew Turk, Luiz Velho
2005 Lecture Notes in Computer Science  
We introduce a novel framework for automatic 3D facial expression analysis in videos. The preliminary results were demonstrated by editing the facial expression with facial recognition. We first build a 3D expression database to learn the expression space of a human face. The real-time 3D video data were captured by a camera/projector scanning system. From this database, we extract the geometry deformation independent of pose and illumination changes. All possible facial deformations of an
more » ... idual make a nonlinear manifold embedded in a high dimensional space. To combine the manifolds of different subjects that vary significantly and are usually hard to align, we transfer the facial deformations in all training videos to one standard model. Lipschitz embedding embeds the normalized deformation of the standard model in a low dimensional generalized manifold. We learn a probabilistic expression model on the generalized manifold. To edit a facial expression of a new subject in 3D videos, the system searches over this generalized manifold for optimal replacement with the 'target' expression, which will be blended with the deformation in the previous frames to synthesize images of the new expression with the current head pose. Experimental results show that our method works effectively
doi:10.1007/11564386_23 fatcat:ikvnrggpdjggne3y66elqllit4