2,165 Hits in 4.6 sec

Video-driven Neural Physically-based Facial Asset for Production [article]

Longwen Zhang, Chuxiao Zeng, Qixuan Zhang, Hongyang Lin, Ruixiang Cao, Wei Yang, Lan Xu, Jingyi Yu
2022 arXiv   pre-print
Comprehensive experiments show that our technique provides higher accuracy and visual fidelity than previous video-driven facial reconstruction and animation methods.  ...  In this paper, we present a new learning-based, video-driven approach for generating dynamic facial geometries with high-quality physically-based assets.  ...  Our learning-based approach generates video-driven neural physically-based facial asset for realistic production.  ... 
arXiv:2202.05592v3 fatcat:tfbmwzburfh7hicvdgwzd4erga

Learning Speech-driven 3D Conversational Gestures from Video [article]

Ikhsanul Habibie, Weipeng Xu, Dushyant Mehta, Lingjie Liu, Hans-Peter Seidel, Gerard Pons-Moll, Mohamed Elgharib, Christian Theobalt
2021 arXiv   pre-print
To this end, we apply state-of-the-art monocular approaches for 3D body and hand pose estimation as well as dense 3D face performance capture to the video corpus.  ...  Our algorithm uses a CNN architecture that leverages the inherent correlation between facial expression and hand gestures.  ...  Dataset Creation Creating 3D Annotations from Video A major bottleneck for previous speech-driven animation synthesis work is the generation of sufficient training data.  ... 
arXiv:2102.06837v1 fatcat:fea24vvphbh4jpuqzclfviq5fe

Everybody's Talkin': Let Me Talk as You Want [article]

Linsen Song, Wayne Wu, Chen Qian, Ran He, Chen Change Loy
2020 arXiv   pre-print
Finally, we introduce a novel video rendering network and a dynamic programming method to construct a temporally coherent and photo-realistic video.  ...  Instead of learning a highly heterogeneous and nonlinear mapping from audio to the video directly, we first factorize each target video frame into orthogonal parameter spaces, i.e., expression, geometry  ...  Face2Face [48] directly transfers facial expression of source video in the parameter space while our method infers facial expression from source audio.  ... 
arXiv:2001.05201v1 fatcat:wes6abhwinghfohufyh46dacwy

2021 Index IEEE Transactions on Multimedia Vol. 23

2021 IEEE transactions on multimedia  
The Author Index contains the primary entry for each item, listed under the first author's name.  ...  ., +, GAC-GAN: A General Method for Appearance-Controllable Human Video Anisotropic Graph Convolutional Network for Semi-Supervised Learning.  ...  ., +, TMM 2021 3306-3317 Anisotropic Graph Convolutional Network for Semi-Supervised Learning.  ... 
doi:10.1109/tmm.2022.3141947 fatcat:lil2nf3vd5ehbfgtslulu7y3lq

Realtime facial animation with on-the-fly correctives

Hao Li, Jihun Yu, Yuting Ye, Chris Bregler
2013 ACM Transactions on Graphics  
Abstract We introduce a real-time and calibration-free facial performance capture framework based on a sensor with video and depth input.  ...  To boost the training of our tracking model with reliable samples, we use a well-trained 2D facial feature tracker on the input video and an efficient mesh deformation algorithm to snap the result of the  ...  , Kirk Haller, Steve Sullivan, and Kim Libreri for their support and supervision.  ... 
doi:10.1145/2461912.2462019 fatcat:7ghpc7pvizh55e6txl4qbamtwe

2020 Index IEEE Transactions on Image Processing Vol. 29

2020 IEEE Transactions on Image Processing  
., +, Self-Enhanced Convolutional Network for Facial Video Hallucination. Fang, C., +, TIP 2020 3078-3090 Self-Supervised Learning of Detailed 3D Face Reconstruction.  ...  ., +, TIP 2020 538-550 Semi-Supervised Robust Mixture Models in RKHS for Abnormality Detection in Medical Images.  ... 
doi:10.1109/tip.2020.3046056 fatcat:24m6k2elprf2nfmucbjzhvzk3m

SliderGAN: Synthesizing Expressive Face Images by Sliding 3D Blendshape Parameters [article]

Evangelos Ververas, Stefanos Zafeiriou
2019 arXiv   pre-print
This provides much more flexibility in various tasks, including but not limited to face editing, expression transfer and face neutralisation, comparing to models based on discrete expressions or action  ...  We show that it is possible to edit a facial image according to expression and speech blendshapes, using sliders that control the continuous values of the blendshape model.  ...  Semi-supervised training We train our model in a semi-supervised manner with both data with no image pairs of the same person under different expressions {I i org , p i org , p i trg } K i=1 and data with  ... 
arXiv:1908.09638v1 fatcat:dz2gbp6urzeqdavqm3b6ktksrm

Challenges and Opportunities for Machine Learning Classification of Behavior and Mental State from Images [article]

Peter Washington, Cezmi Onur Mutlu, Aaron Kline, Kelley Paskov, Nate Tyler Stockham, Brianna Chrisman, Nick Deveau, Mourya Surhabi, Nick Haber, Dennis P. Wall
2022 arXiv   pre-print
While CV classifiers for traditional and structured classification tasks can be developed with standard machine learning pipelines for supervised learning consisting of data labeling, preprocessing, and  ...  training a convolutional neural network, there are several pain points which arise when attempting this process for behavioral phenotyping.  ...  For example, self-supervised learning of facial dynamics in videos has been shown to learn baseline model weights useful for personality prediction [178] .  ... 
arXiv:2201.11197v1 fatcat:fvhzvvctn5drtooemem4u6qloi

Recent Advances in Zero-shot Recognition [article]

Yanwei Fu, Tao Xiang, Yu-Gang Jiang, Xiangyang Xue, Leonid Sigal, and Shaogang Gong
2017 arXiv   pre-print
However, to scale the recognition to a large number of classes with few or now training samples for each class remains an unsolved problem.  ...  With the recent renaissance of deep convolution neural networks, encouraging breakthroughs have been achieved on the supervised recognition tasks, where each class has sufficient training data and fully  ...  Yanwei Fu is supported by The Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning.  ... 
arXiv:1710.04837v1 fatcat:u3mp6dgj2rgqrarjm4dcywegmy

MPEG-4: Audio/video and synthetic graphics/audio for mixed media

Peter K. Doenges, Tolga K. Capin, Fabio Lavagetto, Joern Ostermann, Igor S. Pandzic, Eric D. Petajan
1997 Signal processing. Image communication  
Integrated spatial-temporal coding is sought for audio, video, and 2D/3D computer graphics as standardized A/V objects.  ...  Composition, interactivity, and scripting of A/V objects can thus be supported in client terminals, as well as in content production for servers, also more effectively enabling terminals as servers.  ...  Figure 18 shows another example of facial animation based upon video-driven analysis of facial expressions, with real images and the corresponding results of controlling a synthetic 3D face.  ... 
doi:10.1016/s0923-5965(97)00007-6 fatcat:miw7fareavhbjkjfl7ntjgmoqa

Displaced dynamic expression regression for real-time facial tracking and animation

Chen Cao, Qiming Hou, Kun Zhou
2014 ACM Transactions on Graphics  
Figure 1 : Real-time facial tracking and animation for different users using a single camera.  ...  Abstract We present a fully automatic approach to real-time facial tracking and animation with a single video camera. Our approach does not need any calibration for each individual user.  ...  performers, Steve Lin for proofreading the paper and the SIGGRAPH reviewers for their helpful comments.  ... 
doi:10.1145/2601097.2601204 fatcat:5rz36kzosfcudbjoy7nlfkg7uu

Real-Time Facial Segmentation and Performance Capture from RGB Input [article]

Shunsuke Saito, Tianye Li, Hao Li
2016 arXiv   pre-print
To ensure robustness, cutting edge supervised learning approaches rely on large training datasets of face images captured in the wild.  ...  Along with recent breakthroughs in deep learning, we demonstrate that pixel-level facial segmentation is possible in real-time by repurposing convolutional neural networks designed originally for general  ...  We also thank Rui Saito and Frances Chen for being our capture models.  ... 
arXiv:1604.02647v1 fatcat:i3bdzgot4ffixn4jcm7hqyb7ze

Structure-aware Editable Morphable Model for 3D Facial Detail Animation and Manipulation [article]

Jingwang Ling, Zhibo Wang, Ming Lu, Quan Wang, Chen Qian, Feng Xu
2022 arXiv   pre-print
Morphable models are essential for the statistical modeling of 3D faces. Previous works on morphable models mostly focus on large-scale facial geometry but ignore facial details.  ...  Extensive experiments demonstrate that the proposed model compactly represents facial details, outperforms previous methods in expression animation qualitatively and quantitatively, and achieves effective  ...  For animation results, please refer to our supplementary video.  ... 
arXiv:2207.09019v1 fatcat:ihdrmubnqvdfxdnztk7ogwtwv4

The Creation and Detection of Deepfakes: A Survey [article]

Yisroel Mirsky, Wenke Lee
2020 arXiv   pre-print
In 2018, it was discovered how easy it is to use this technology for unethical and malicious applications, such as the spread of misinformation, impersonation of political leaders, and the defamation of  ...  A multi-scale loss is used to improve quality and the authors utilize a small labeled dataset by training their model in a semi-supervised way.  ...  In contrast, the authors of [144] propose Monkey-Net: a self supervised network for driving an image with an arbitrary video sequence.  ... 
arXiv:2004.11138v3 fatcat:xqabyslmdfhyznm7msqp3wznnq

SliderGAN: Synthesizing Expressive Face Images by Sliding 3D Blendshape Parameters

Evangelos Ververas, Stefanos Zafeiriou
2020 International Journal of Computer Vision  
This provides much more flexibility in various tasks, including but not limited to face editing, expression transfer and face neutralisation, comparing to models based on discrete expressions or action  ...  We show that it is possible to edit a facial image according to expression and speech blendshapes, using sliders that control the continuous values of the blendshape model.  ...  Additionally, we would like to thank the reviewers for their valuable comments that helped us to improve this paper.  ... 
doi:10.1007/s11263-020-01338-7 fatcat:j2ml4p3dwvhk5gzvekbptoryma
« Previous Showing results 1 — 15 out of 2,165 results