Real-time 3D face tracking based on active appearance model constrained by depth data

Nikolai Smolyanskiy, Christian Huitema, Lin Liang, Sean Eron Anderson
2014 Image and Vision Computing  
Active Appearance Model (AAM) is an algorithm for fitting a generative model of object shape and appearance to an input image. AAM allows accurate, real-time tracking of human faces in 2D and can be extended to track faces in 3D by constraining its fitting with a linear 3D morphable model. Unfortunately, this AAM-based 3D tracking does not provide adequate accuracy and robustness, as we show in this paper. We introduce a new constraint into AAM fitting that uses depth data from a commodity RGBD
more » ... camera (Kinect). This addition significantly reduces 3D tracking errors. We also describe how to initialize the 3D morphable face model used in our tracking algorithm by computing its face shape parameters of the user from a batch of tracked frames. The described face tracking algorithm is used in Microsoft's Kinect system.
doi:10.1016/j.imavis.2014.08.005 fatcat:4pd4jdvkxrdgxmnxw2u74chmcy