Full Face-and-Head 3D Model with Photorealistic Texture

Yangyu Fan, Yang Liu, Guoyun Lv, Shiya Liu, Gen Li, Yanhui Huang
2020 IEEE Access  
In the recent period, significant progress has been achieved towards reconstructing the 3D face model from face image. With the support of the render engines and sufficient data, the reconstruction results are fine in detail. Nevertheless, the research on the 3D face reconstruction with texture from a single unrestricted face image is imperfect. The rebuild process lacks essential structure and texture information in the profile and the craniofacial region. To address this problem, we present a
more » ... method of creating a 3D full face-and-head model with photorealistic texture from a single "in-the-wild" face image in this paper. To this end, we introduce a pipeline to integrate the highly-detailed face model into the basic model. Specifically, the basic model was built by multilinear optimization, and the highly-detailed face model which represents the facial features generated by constrained illumination distribution. Additionally, to infer the invisible region texture information corresponding to the input face image, we design an effective architecture with the generative adversarial network (GAN) for panoramic UV texture generation. The final results after UV texture mapping were visualized in the experiment, which demonstrates that the model faithfully recovers the photorealistic details in arbitrary perspective. Furthermore, compared to the state-of-the-art facial modeling techniques and existing commercial solutions, our method takes less input and performs better in surface detail. INDEX TERMS 3D face reconstruction, full face-and-head model, 3D Morphable Model, UV texture, Generative Adversarial Networks. VOLUME XX, 2020 1 FIGURE 1. Overview of our proposed method. We build the basic model by joint optimizing of the input images and the database and integrate it with the highly-detailed model to obtain the full face-and-head mesh model. The UV texture can be generated by our network architecture with a single "inthe-wild" face image. The complete full face-and-head model is acquired after the texture mapping and the component completion.
doi:10.1109/access.2020.3031886 fatcat:zzwmtc5ugbc75i5ln5xf66d3zy