APB2Face: Audio-guided face reenactment with auxiliary pose and blink signals [article]

Jiangning Zhang, Liang Liu, Zhucun Xue, Yong Liu
2020 arXiv   pre-print
Audio-guided face reenactment aims at generating photorealistic faces using audio information while maintaining the same facial movement as when speaking to a real person. However, existing methods can not generate vivid face images or only reenact low-resolution faces, which limits the application value. To solve those problems, we propose a novel deep neural network named APB2Face, which consists of GeometryPredictor and FaceReenactor modules. GeometryPredictor uses extra head pose and blink
more » ... tate signals as well as audio to predict the latent landmark geometry information, while FaceReenactor inputs the face landmark image to reenact the photorealistic face. A new dataset AnnVI collected from YouTube is presented to support the approach, and experimental results indicate the superiority of our method than state-of-the-arts, whether in authenticity or controllability.
arXiv:2004.14569v1 fatcat:uqogtp4f35avpbqpmo3cqgxxgi