Facial Performance Transfer via Deformable Models and Parametric Correspondence

A. Asthana, M. de la Hunty, A. Dhall, R. Goecke
2012 IEEE Transactions on Visualization and Computer Graphics  
The issue of transferring facial performance from one person's face to another's has been an area of interest for the movie industry and the computer graphics community for quite some time. In recent years, deformable face models, such as the Active Appearance Model (AAM), have made it possible to track and synthesise faces in real-time. Not surprisingly, deformable face model based approaches for facial performance transfer have gained tremendous interest in the computer vision and graphics
more » ... ion and graphics community. In this paper, we focus on the problem of real-time facial performance transfer using the AAM framework. We propose a novel approach of learning the mapping between the parameters of two completely independent AAMs, using them to facilitate the facial performance transfer in a more realistic manner than previous approaches. The main advantage of modelling this parametric correspondence is that it allows a "meaningful" transfer of both the non-rigid shape and texture across faces irrespective of the speakers' gender, shape and size of the faces, and illumination conditions. We explore linear and non-linear methods for modelling the parametric correspondence between the AAMs and show that the sparse linear regression method performs the best. Moreover, we show the utility of the proposed framework for a cross-language facial performance transfer that is an area of interest for the movie dubbing industry.
doi:10.1109/tvcg.2011.157 pmid:21931176 fatcat:ab6cqgs5uveyrh7o7tpayoxko4