Filters








383 Hits in 6.3 sec

High-Fidelity Neural Human Motion Transfer from Monocular Video

Moritz Kappel, Vladislav Golyanik, Mohamed Elgharib, Jann-Ole Henningson, Hans-Peter Seidel, Susana Castillo, Christian Theobalt, Marcus Magnor
2021 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
High-Fidelity Neural Human Motion Transfer from Monocular Video -Supplementary Material - In this appendix, we provide more details on the methods examined in the main paper along with additional results  ...  Conclusion We introduced a novel multi-stage framework for highfidelity human motion transfer from monocular video.  ... 
doi:10.1109/cvpr46437.2021.00159 fatcat:yxsdurntwjhpba2kw45vw2yx5m

Robust Pose Transfer with Dynamic Details using Neural Video Rendering [article]

Yang-tian Sun, Hao-zhi Huang, Xuan Wang, Yu-kun Lai, Wei Liu, Lin Gao
2021 arXiv   pre-print
Pose transfer of human videos aims to generate a high fidelity video of a target person imitating actions of a source person.  ...  In this paper, we demonstrate that the dynamic details can be preserved even trained from short monocular videos.  ...  INTRODUCTION R ECENTLY, great progress has been achieved by applying neural networks to video synthesis, especially the human motion transfer task, which aims to transfer the action of a source person  ... 
arXiv:2106.14132v2 fatcat:dslhnmaborfstasn2qh6lwejly

SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video [article]

Boyi Jiang, Yang Hong, Hujun Bao, Juyong Zhang
2022 arXiv   pre-print
We propose SelfRecon, a clothed human body reconstruction method that combines implicit and explicit representations to recover space-time coherent geometries from a monocular self-rotating human video  ...  Compared with existing methods, SelfRecon can produce high-fidelity surfaces for arbitrary clothed humans with self-supervised optimization.  ...  In this work, we propose SelfRecon, which combines the explicit and implicit representations together to reconstruct high-fidelity digital avatar from a monocular video.  ... 
arXiv:2201.12792v2 fatcat:ocubpi7ug5glxonubsl7crluhe

Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation [article]

Lingjie Liu, Weipeng Xu, Marc Habermann, Michael Zollhoefer, Florian Bernard, Hyeongwoo Kim, Wenping Wang, Christian Theobalt
2021 arXiv   pre-print
Synthesizing realistic videos of humans using neural networks has been a popular alternative to the conventional graphics-based rendering pipeline due to its high efficiency.  ...  We demonstrate several applications of our approach, such as human reenactment and novel view synthesis from monocular video, where we show significant improvement over the state of the art both qualitatively  ...  , i.e., to transfer the motion from the source video to the target video.  ... 
arXiv:2001.04947v3 fatcat:ppii2ilexze7nkejshrohlky4u

Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera [article]

Jae Shin Yoon, Duygu Ceylan, Tuanfeng Y. Wang, Jingwan Lu, Jimei Yang, Zhixin Shu, Hyun Soo Park
2022 arXiv   pre-print
This learned representation is decoded by a compositional multi-task decoder that renders high fidelity time-varying appearance.  ...  Our experiments show that our method can generate a temporally coherent video of dynamic humans for unseen body poses and novel views given a single view video.  ...  Jae Shin Yoon is supported by Doctoral Dissertation Fellowship from University of Minnesota. This work is partially supported by NSF CNS-1919965.  ... 
arXiv:2203.12780v1 fatcat:i2o7bdrgrrav5ajox6i6nzj3xe

Neural Rendering and Reenactment of Human Actor Videos [article]

Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, Christian Theobalt
2019 arXiv   pre-print
We evaluate our method for the reenactment of another person that is tracked in order to obtain the motion data, and show video results generated from artist-designed skeleton motion.  ...  Technically, this is achieved by training a neural network that translates simple synthetic images of a human character into realistic imagery.  ...  Note that the source motion can either come from a tracked monocular video, or from any other motion source (e.g., user-defined motion, or MoCap data). for the network comprise individual depth and color  ... 
arXiv:1809.03658v3 fatcat:ciqgsmd3vffbvj3tye474gpxre

Neural Re-Rendering of Humans from a Single Image [article]

Kripasindhu Sarkar, Dushyant Mehta, Weipeng Xu, Vladislav Golyanik, Christian Theobalt
2021 arXiv   pre-print
To address these challenges, we propose a new method for neural re-rendering of a human under a novel user-defined pose and viewpoint, given one input image.  ...  Human re-rendering from a single image is a starkly under-constrained problem, and state-of-the-art algorithms often exhibit undesired artefacts, such as over-smoothing, unrealistic distortions of the  ...  Neural Re-Rendering of Humans from a Single Image  ... 
arXiv:2101.04104v1 fatcat:6snujhlbpbgd7nc4gcw62soivy

Neural Rendering and Reenactment of Human Actor Videos

Lingjie Liu, Weipeng Xu, Michael Zollhöfer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, Christian Theobalt
2019 ACM Transactions on Graphics  
We evaluate our method for the reenactment of another person that is tracked in order to obtain the motion data, and show video results generated from artist-designed skeleton motion.  ...  Our results outperform the state-of-the-art in learning-based human image synthesis.  ...  Note that the source motion can either come from a tracked monocular video, or from any other motion source (e.g., user-defined motion, or MoCap data). for the network comprise individual depth and color  ... 
doi:10.1145/3333002 fatcat:l2ceefpzj5crfmtnu3hcqpuzdy

Video-driven Neural Physically-based Facial Asset for Production [article]

Longwen Zhang, Chuxiao Zeng, Qixuan Zhang, Hongyang Lin, Ruixiang Cao, Wei Yang, Lan Xu, Jingyi Yu
2022 arXiv   pre-print
We demonstrate our approach in high-fidelity performer-specific facial capture and cross-identity facial motion retargeting.  ...  Production-level workflows for producing convincing 3D dynamic human faces have long relied on an assortment of labor-intensive tools for geometry and texture generation, motion capture and rigging, and  ...  We demonstrate our approach in high-fidelity performer-specific facial capture and cross-identity facial motion transfer and retargeting.  ... 
arXiv:2202.05592v4 fatcat:t5zdwsnvcfe4ncb5ceibyhtzpu

Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes: The Importance of Multiple Scene Constraints

Andrei Zanfir, Elisabeta Marinoiu, Cristian Sminchisescu
2018 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition  
In this paper, we leverage state-of-the-art deep multi-task neural networks and parametric human and scene modeling, towards a fully automatic monocular visual sensing system for multiple interacting people  ...  Human sensing has greatly benefited from recent advances in deep learning, parametric human modeling, and large scale 2d and 3d datasets.  ...  or acceleration, or more sophisticated models learned from human motion capture data.  ... 
doi:10.1109/cvpr.2018.00229 dblp:conf/cvpr/ZanfirMS18 fatcat:ooi7gxd6c5d35gaumk4uuuparu

AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis [article]

Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, Juyong Zhang
2021 arXiv   pre-print
Specifically, the feature of input audio signal is directly fed into a conditional implicit function to generate a dynamic neural radiance field, from which a high-fidelity talking-head video corresponding  ...  Generating high-fidelity talking head video by fitting with the input audio sequence is a challenging problem that receives considerable attentions recently.  ...  Introduction Synthesizing high-fidelity audio-driven facial video sequences is an important and challenging problem in many applications like digital humans, chatting robots, and virtual video conferences  ... 
arXiv:2103.11078v3 fatcat:jcyv42gednfszjqxymkqojdaxe

DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenes [article]

Jia-Wei Liu, Yan-Pei Cao, Weijia Mao, Wenqiao Zhang, David Junhao Zhang, Jussi Keppo, Ying Shan, Xiaohu Qie, Mike Zheng Shou
2022 arXiv   pre-print
Despite achieving unprecedented fidelity for novel view synthesis in dynamic scenes, existing methods based on Neural Radiance Fields (NeRF) suffer from slow convergence (i.e., model training time measured  ...  Experiments demonstrate that DeVRF achieves two orders of magnitude speedup (100x faster) with on-par high-fidelity results compared to the previous state-of-the-art approaches.  ...  As a result, subsequent studies [25, 26, 17, 7] only capture forward-facing videos of real-world dynamic scenes with a monocular camera.  ... 
arXiv:2205.15723v2 fatcat:c2d2golx3naq7mfs6ofzkvfieq

VR content creation and exploration with deep learning: A survey

Miao Wang, Xu-Quan Lyu, Yi-Jun Li, Fang-Lue Zhang
2020 Computational Visual Media  
employed, designed specifically to handle panoramic images and video and virtual 3D scenes.  ...  VR content creation and exploration relates to image and video analysis, synthesis and editing, so deep learning methods such as fully convolutional networks and general adversarial networks are widely  ...  [98] presented a recurrent neural network to predict parts and motion from point clouds.  ... 
doi:10.1007/s41095-020-0162-z fatcat:lgogzx26bvhn5f7uyefjkz7zny

LASR: Learning Articulated Shape Reconstruction from a Monocular Video [article]

Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T. Freeman, Ce Liu
2021 arXiv   pre-print
Without using a category-specific shape template, our method faithfully reconstructs nonrigid 3D structures from videos of human, animals, and objects of unknown classes.  ...  and motion parameters.  ...  By relying on generic shape and motion priors, LASR applies to a wider range of nonrigid shapes and yields high-fidelity 3D reconstructions: It recovers both humps of the camel, which are missing from  ... 
arXiv:2105.02976v1 fatcat:oostakof65ah7epconiy63y24y

EgoRenderer: Rendering Human Avatars from Egocentric Camera Images [article]

Tao Hu, Kripasindhu Sarkar, Lingjie Liu, Matthias Zwicker, Christian Theobalt
2021 arXiv   pre-print
Our system renders photorealistic novel views of the actor and her motion from arbitrary virtual camera locations.  ...  For texture synthesis, we propose Ego-DPNet, a neural network that infers dense correspondences between the input fisheye images and an underlying parametric body model, and to extract textures from egocentric  ...  Similarly, recent neural rendering-based pose transfer methods enable creation of highly realistic animation videos of humans under user-specified target motion [5, 2, 21, 54] .  ... 
arXiv:2111.12685v1 fatcat:ibpmcwfv2zclzmdvc7ivt26kfy
« Previous Showing results 1 — 15 out of 383 results