Filters








275 Hits in 4.9 sec

Neural Rendering and Reenactment of Human Actor Videos [article]

Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, Christian Theobalt
2019 arXiv   pre-print
For training our networks, we first track the 3D motion of the person in the video using the template model, and subsequently generate a synthetically rendered version of the video.  ...  We evaluate our method for the reenactment of another person that is tracked in order to obtain the motion data, and show video results generated from artist-designed skeleton motion.  ...  This work was supported by ERC Consolidator Grant 4DRepLy (770784), Max Planck Center for Visual Computing and Communications (MPC-VCC) and ITC of Hong Kong (ITS/457/17FP).  ... 
arXiv:1809.03658v3 fatcat:ciqgsmd3vffbvj3tye474gpxre

Neural Rendering and Reenactment of Human Actor Videos

Lingjie Liu, Weipeng Xu, Michael Zollhöfer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, Christian Theobalt
2019 ACM Transactions on Graphics  
We evaluate our method for the reenactment of another person that is tracked in order to obtain the motion data, and show video results generated from artist-designed skeleton motion.  ...  Our results outperform the state-of-the-art in learning-based human image synthesis.  ...  This work was supported by ERC Consolidator Grant 4DRepLy (770784), Max Planck Center for Visual Computing and Communications (MPC-VCC) and ITC of Hong Kong (ITS/457/17FP).  ... 
doi:10.1145/3333002 fatcat:l2ceefpzj5crfmtnu3hcqpuzdy

ReenactNet: Real-time Full Head Reenactment [article]

Mohammad Rami Koujan, Michail Christos Doukas, Anastasios Roussos, Stefanos Zafeiriou
2020 arXiv   pre-print
We propose a head-to-head system of our own implementation capable of fully transferring the human head 3D pose, facial expressions and eye gaze from a source to a target actor, while preserving the identity  ...  of the target actor.  ...  The proposed head2head pipeline consists of two successive stages: 1) 3D facial reconstruction and tracking, followed by 2) learningbased video rendering, using a neural network.  ... 
arXiv:2006.10500v1 fatcat:6bfydlrvcbbvtdcckpbs6emcl4

Neural Relighting and Expression Transfer On Video Portraits [article]

Youjia Wang, Taotao Zhou, Minzhang Li, Teng Xu, Minye Wu, Lan Xu, Jingyi Yu
2021 arXiv   pre-print
Photo-realistic video portrait reenactment benefits virtual production and numerous VR/AR experiences.  ...  Our approach employs 4D reflectance field learning, model-based facial performance capture and target-aware neural rendering.  ...  Face reenactment and replacement. Facial reenactment re-generates the face content of a target actor in a portrait video by transferring facial expression and pose from a source actor.  ... 
arXiv:2107.14735v4 fatcat:nd7bvngaijhm7n57d372sykafi

Head2Head: Video-based Neural Head Synthesis [article]

Mohammad Rami Koujan, Michail Christos Doukas, Anastasios Roussos, Stefanos Zafeiriou
2020 arXiv   pre-print
We demonstrate that the proposed method can transfer facial expressions, pose and gaze of a source actor to a target video in a photo-realistic fashion more accurately than state-of-the-art methods.  ...  exploits the special structure of facial motion (paying particular attention to mouth motion) and (b) enforces temporal consistency.  ...  Zafeiriou acknowledges funding from a Google Faculty Award, as well as from the EPSRC Fellowship DEFORM: Large Scale Shape Analysis of Deformable Models of Humans (EP/S010203/1).  ... 
arXiv:2005.10954v1 fatcat:kw43v65lindf3oapvw7rnjxgbu

Neural Voice Puppetry: Audio-driven Facial Reenactment [article]

Justus Thies, Mohamed Elgharib, Ayush Tewari, Christian Theobalt, Matthias Nießner
2020 arXiv   pre-print
Neural Voice Puppetry has a variety of use-cases, including audio-driven video avatars, video dubbing, and text-driven video synthesis of a talking head.  ...  Our approach generalizes across different people, allowing us to synthesize videos of a target actor with the voice of any unknown source actor or even synthetic voices that can be generated utilizing  ...  The 'Deferred Neural Rendering' [33] is a generic neural rendering approach, but the authors also show the usage in the scenario of facial reenactment.  ... 
arXiv:1912.05566v2 fatcat:sazmvejurrbu7kadsdjgejc2am

High-Fidelity Neural Human Motion Transfer from Monocular Video

Moritz Kappel, Vladislav Golyanik, Mohamed Elgharib, Jann-Ole Henningson, Hans-Peter Seidel, Susana Castillo, Christian Theobalt, Marcus Magnor
2021 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
We thank Jalees Nehvi and Navami Kairanda for helping with comparisons.  ...  Although our method is a step forward in the neural rendering of humans, it has limitations and can provoke further research.  ...  Textured Neural Avatars [32] can be trained for rendering a target actor in tight clothes in arbitrary body and camera poses.  ... 
doi:10.1109/cvpr46437.2021.00159 fatcat:yxsdurntwjhpba2kw45vw2yx5m

Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation [article]

Lingjie Liu, Weipeng Xu, Marc Habermann, Michael Zollhoefer, Florian Bernard, Hyeongwoo Kim, Wenping Wang, Christian Theobalt
2021 arXiv   pre-print
Synthesizing realistic videos of humans using neural networks has been a popular alternative to the conventional graphics-based rendering pipeline due to its high efficiency.  ...  We demonstrate several applications of our approach, such as human reenactment and novel view synthesis from monocular video, where we show significant improvement over the state of the art both qualitatively  ...  To the best of our knowledge, our approach is the first dynamic-texture neural rendering approach for human bodies that disentangles human video synthesis into explicit texture-space and image-space neural  ... 
arXiv:2001.04947v3 fatcat:ppii2ilexze7nkejshrohlky4u

Deferred Neural Rendering: Image Synthesis using Neural Textures [article]

Justus Thies and Michael Zollhöfer and Matthias Nießner
2019 arXiv   pre-print
This way, neural textures can be utilized to coherently re-render or manipulate existing video content in both static and dynamic environments at real-time rates.  ...  We show the effectiveness of our approach in several experiments on novel view synthesis, scene editing, and facial reenactment, and compare to state-of-the-art approaches that leverage the standard graphics  ...  Reenactment can be done with any new source actor.  ... 
arXiv:1904.12356v1 fatcat:r3vfiidi4fdgpnbr6phlvhsuuu

Deep Semantic Manipulation of Facial Videos [article]

Girish Kumar Solanki, Anastasios Roussos
2021 arXiv   pre-print
Our method supports semantic video manipulation based on neural rendering and 3D-based facial expression modelling.  ...  We also introduce a user-friendly, interactive AI tool that processes human-readable semantic labels about the desired emotion manipulations in specific parts of the input video and synthesizes photorealistic  ...  In addition, we build upon state-of-the-art neural rendering methods for face and head reenactment [13] [14] , achieving photorealistic videos with manipulated expressions.  ... 
arXiv:2111.07902v1 fatcat:q7bc6r6ryvaibif4zrfyhytu4q

Generative Video Face Reenactment by AUs and Gaze Regularization

Josep Famadas, Meysam Madadi, Cristina Palmero, Sergio Escalera
2020 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)  
At test stage, a new target subject with the facial performance of the source subject and the appearance of the training subject is reenacted.  ...  We regularize face reenactment by facial action unit intensity and 3D gaze vector regression.  ...  These parameters are used to render the synthetic images that are grouped by sequence and fed into a rendering-to-video network.  ... 
doi:10.1109/fg47880.2020.00125 fatcat:b7grtnnnlrbvzp7sx6kkjekowm

Head2Head++: Deep Facial Attributes Re-Targeting [article]

Michail Christos Doukas, Mohammad Rami Koujan, Viktoriia Sharmanska, Anastasios Roussos
2020 arXiv   pre-print
We leverage the 3D geometry of faces and Generative Adversarial Networks (GANs) to design a novel deep learning architecture for the task of facial and head reenactment.  ...  source video to a target subject, in a photo-realistic and faithful fashion, better than other state-of-the-art methods.  ...  , neural renderer.  ... 
arXiv:2006.10199v1 fatcat:ylapnwbkzjes5gejb4zmnou6ym

The Creation and Detection of Deepfakes: A Survey [article]

Yisroel Mirsky, Wenke Lee
2020 arXiv   pre-print
In this paper, we explore the creation and detection of deepfakes and provide an in-depth view of how these architectures work.  ...  In 2018, it was discovered how easy it is to use this technology for unethical and malicious applications, such as the spread of misinformation, impersonation of political leaders, and the defamation of  ...  and by state actors to perform espionage and sabotage by reenacting of o cials or family members.  ... 
arXiv:2004.11138v3 fatcat:xqabyslmdfhyznm7msqp3wznnq

A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos [article]

Thiago L. Gomes and Renato Martins and João Ferreira and Rafael Azevedo and Guilherme Torres and Erickson R. Nascimento
2021 arXiv   pre-print
Transferring human motion and appearance between videos of human actors remains one of the key challenges in Computer Vision.  ...  In this work, we propose a shape-aware approach based on a hybrid image-based rendering technique that exhibits competitive visual retargeting quality compared to state-of-the-art neural rendering approaches  ...  The authors thank CAPES, CNPq, and FAPEMIG for funding this work. We also thank NVIDIA for the donation of a Titan XP GPU used in this research.  ... 
arXiv:2103.15596v2 fatcat:them7wrlvfchpgy36mtxmfzm2q

FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality [article]

Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, Matthias Nießner
2018 arXiv   pre-print
Based on reenactment of a prerecorded stereo video of the person without the HMD, FaceVR incorporates photo-realistic re-rendering in real time, thus allowing artificial modifications of face and eye appearances  ...  We propose FaceVR, a novel image-based method that enables video teleconferencing in VR based on self-reenactment.  ...  ACKNOWLEDGMENTS We thank Angela Dai for the video voice over and all actors for the VR reenactment. The facial landmark tracker was kindly provided by TrueVisionSolution.  ... 
arXiv:1610.03151v2 fatcat:pnx2nfjqrrgpbcpaarsrg6ykwi
« Previous Showing results 1 — 15 out of 275 results