Filters








132 Hits in 5.1 sec

Realistic Face Reenactment via Self-Supervised Disentangling of Identity and Pose [article]

Xianfang Zeng, Yusu Pan, Mengmeng Wang, Jiangning Zhang, Yong Liu
2020 arXiv   pre-print
To alleviate the demand for manual annotations, in this paper, we propose a novel self-supervised hybrid model (DAE-GAN) that learns how to reenact face naturally given large amounts of unlabeled videos  ...  On the one hand, we adopt the deforming autoencoder to disentangle identity and pose representations.  ...  Through the disentanglement of identity and pose representations, our model is able to reenact face naturally between different identities. Self-supervised representation disentanglement.  ... 
arXiv:2003.12957v1 fatcat:ztnbksxinvd7nc4lifyhbav7nq

Realistic Face Reenactment via Self-Supervised Disentangling of Identity and Pose

Xianfang Zeng, Yusu Pan, Mengmeng Wang, Jiangning Zhang, Yong Liu
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
To alleviate the demand for manual annotations, in this paper, we propose a novel self-supervised hybrid model (DAE-GAN) that learns how to reenact face naturally given large amounts of unlabeled videos  ...  On the one hand, we adopt the deforming autoencoder to disentangle identity and pose representations.  ...  Through the disentanglement of identity and pose representations, our model is able to reenact face naturally between different identities. Self-supervised representation disentanglement.  ... 
doi:10.1609/aaai.v34i07.6970 fatcat:ivymkl4nm5ghjjlqimxplzxoya

Task-agnostic Temporally Consistent Facial Video Editing [article]

Meng Cao, Haozhi Huang, Hao Wang, Xuan Wang, Li Shen, Sheng Wang, Linchao Bao, Zhifeng Li, Jiebo Luo
2020 arXiv   pre-print
Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.  ...  Based on a 3D reconstruction model, our framework is designed to handle several editing tasks in a more unified and disentangled manner.  ...  It means that X t i , X t p , and X t e have the same identity, which forms self-supervision.  ... 
arXiv:2007.01466v1 fatcat:ya3ka7jmlnakrkbyrdbdupxgpi

UniFaceGAN: A Unified Framework for Temporally Consistent Facial Video Editing [article]

Meng Cao, Haozhi Huang, Hao Wang, Xuan Wang, Li Shen, Sheng Wang, Linchao Bao, Zhifeng Li, Jiebo Luo
2021 arXiv   pre-print
Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.  ...  Based on a 3D reconstruction model and a simple yet efficient dynamic training sample selection mechanism, our framework is designed to handle face swapping and face reenactment simultaneously.  ...  Monkey-Net [12] aims to encode motion information via moving keypoints learned in a self-supervised fashion.  ... 
arXiv:2108.05650v1 fatcat:df5o7n4panc5voqcn5rlr2htjy

FACEGAN: Facial Attribute Controllable rEenactment GAN [article]

Soumya Tripathy, Juho Kannala, Esa Rahtu
2020 arXiv   pre-print
The face reenactment is a popular facial animation method where the person's identity is taken from the source image and the facial motion from the driving image.  ...  We propose a novel Facial Attribute Controllable rEenactment GAN (FACEGAN), which transfers the facial motion from the driving face via the Action Unit (AU) representation.  ...  The MSE score measures the pixel-wise differences in the self-reenactment case. on the face identity and the driving landmarks directly provide strong supervision in these terms.  ... 
arXiv:2011.04439v1 fatcat:lqkpt2zor5hmjbgvpfrej2gnue

ICface: Interpretable and Controllable Face Reenactment Using GANs

Soumya Tripathy, Juho Kannala, Esa Rahtu
2020 2020 IEEE Winter Conference on Applications of Computer Vision (WACV)  
Abstract This paper presents a generic face animator that is able to control the pose and expressions of a given face image.  ...  The proposed face animator is implemented as a two stage neural network model that is learned in self-supervised manner using a large video collection.  ...  Each Face Reenactment In face reenactment, the task is to generate a realistic face image depicting a given source person with the same pose and expression as in a given driving face image.  ... 
doi:10.1109/wacv45572.2020.9093474 dblp:conf/wacv/TripathyKR20 fatcat:wntrwugusrbx7idxqlfkebmmoe

The Creation and Detection of Deepfakes: A Survey [article]

Yisroel Mirsky, Wenke Lee
2020 arXiv   pre-print
In this paper, we explore the creation and detection of deepfakes and provide an in-depth view of how these architectures work.  ...  In 2018, it was discovered how easy it is to use this technology for unethical and malicious applications, such as the spread of misinformation, impersonation of political leaders, and the defamation of  ...  Later in 2019, the authors of [175] proposed a self-supervised network which can change the identity of an object within an image. eir ED disentangles the identity from an objects pose using a novel  ... 
arXiv:2004.11138v3 fatcat:xqabyslmdfhyznm7msqp3wznnq

ICface: Interpretable and Controllable Face Reenactment Using GANs [article]

Soumya Tripathy, Juho Kannala, Esa Rahtu
2020 arXiv   pre-print
This paper presents a generic face animator that is able to control the pose and expressions of a given face image.  ...  The proposed face animator is implemented as a two-stage neural network model that is learned in a self-supervised manner using a large video collection.  ...  Each Face Reenactment In face reenactment, the task is to generate a realistic face image depicting a given source person with the same pose and expression as in a given driving face image.  ... 
arXiv:1904.01909v2 fatcat:amjfw3qlgzdindv2vb55zucaj4

MegaPortraits: One-shot Megapixel Neural Head Avatars [article]

Nikita Drobyshev, Jenya Chelishev, Taras Khakhulin, Aleksei Ivakhnenko, Victor Lempitsky, Egor Zakharov
2022 arXiv   pre-print
Real-time operation and identity lock are essential for many practical applications head avatar systems.  ...  Lastly, we show how a trained high-resolution neural avatar model can be distilled into a lightweight student model which runs in real-time and locks the identities of neural avatars to several dozens  ...  ACKNOWLEDGEMENTS We sincerely thank Michail Christos Doukas for providing us inference results of HeadGAN [8] system and Ting-Chun Wang for providing us inference results of Face-V2V [34] system.  ... 
arXiv:2207.07621v1 fatcat:omvp3qns5zg7vhb4bedw6r7bme

Head2Head++: Deep Facial Attributes Re-Targeting [article]

Michail Christos Doukas, Mohammad Rami Koujan, Viktoriia Sharmanska, Anastasios Roussos
2020 arXiv   pre-print
We leverage the 3D geometry of faces and Generative Adversarial Networks (GANs) to design a novel deep learning architecture for the task of facial and head reenactment.  ...  source video to a target subject, in a photo-realistic and faithful fashion, better than other state-of-the-art methods.  ...  For that, we synthesised five videos of 75 frames (3 seconds) each, using different reenactment methods (head, face and self reenactment).  ... 
arXiv:2006.10199v1 fatcat:ylapnwbkzjes5gejb4zmnou6ym

Deep Semantic Manipulation of Facial Videos [article]

Girish Kumar Solanki, Anastasios Roussos
2021 arXiv   pre-print
Editing and manipulating facial features in videos is an interesting and important field of research with a plethora of applications, ranging from movie post-production and visual effects to realistic  ...  The proposed method is based on a disentangled representation and estimation of the 3D facial shape and activity, providing the user with intuitive and easy-to-use control of the facial expressions in  ...  The process involves the estimation of facial landmarks and eye pupils, as well as disentangled components of the 3D face (identity, expressions, pose), which helps us effectively modify the expression  ... 
arXiv:2111.07902v1 fatcat:q7bc6r6ryvaibif4zrfyhytu4q

Depth-Aware Generative Adversarial Network for Talking Head Video Generation [article]

Fa-Ting Hong, Longhao Zhang, Li Shen, Dan Xu
2022 arXiv   pre-print
Talking head video generation aims to produce a synthetic human face video that contains the identity and pose information respectively from a given source image and a driving video.Existing works for  ...  In this paper, we first introduce a self-supervised geometry learning method to automatically recover the dense 3D geometry (i.e.depth) from the face videos without the requirement of any expensive 3D  ...  More than that, data augmentation strategies [1, 40] are also explored to more effectively perform the disentanglement of the pose and identity information.  ... 
arXiv:2203.06605v2 fatcat:5xzosxgqwretre4md7e6z22jrm

Face Identity Disentanglement via Latent Space Mapping [article]

Yotam Nitzan, Amit Bermano, Yangyan Li, Daniel Cohen-Or
2020 arXiv   pre-print
Through extensive experimentation, we show that our method successfully disentangles identity from other facial attributes, surpassing existing methods, even though they require more training and supervision  ...  Learning disentangled representations of data is a fundamental problem in artificial intelligence.  ...  This work was supported by the Israel Science Foundation (grant no. 2366/16 and 2472/17) and in part by the National Science Foundation of China General Program grant No. 61772317  ... 
arXiv:2005.07728v3 fatcat:3jbzg6peq5h2hgo7p67sgivmre

PIE: Portrait Image Embedding for Semantic Control [article]

Ayush Tewari, Mohamed Elgharib, Mallikarjun B R., Florian Bernard, Hans-Peter Seidel, Patrick Pérez, Michael Zollhöfer, Christian Theobalt
2020 arXiv   pre-print
We present the first approach for embedding real portrait images in the latent space of StyleGAN, which allows for intuitive editing of the head pose, facial expression, and scene illumination in the image  ...  For ease of use, control should be provided via a semantically meaningful parameterization that is akin to computer animation controls.  ...  ACKNOWLEDGMENTS We thank True-VisionSolutions Pty Ltd for providing the 2D face tracker.  ... 
arXiv:2009.09485v1 fatcat:twv7rw3w4baizj3nomqbvxlluq

PIE

Ayush Tewari, Mohamed Elgharib, Mallikarjun B R, Florian Bernard, Hans-Peter Seidel, Patrick Pérez, Michael Zollhöfer, Christian Theobalt
2020 ACM Transactions on Graphics  
We evaluate our approach on a wide set of portrait photos, compare it to the current state of the art, and validate the effectiveness of its components in an ablation study.  ...  Our approach runs at interactive frame rates and thus allows the user to explore the space of possible edits.  ...  ACKNOWLEDGMENTS We thank True-VisionSolutions Pty Ltd for providing the 2D face tracker.  ... 
doi:10.1145/3414685.3417803 fatcat:ezhpt4ecwndqxeoh54g4fpmoiy
« Previous Showing results 1 — 15 out of 132 results