1,860 Hits in 5.0 sec

Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation [article]

Lingjie Liu, Weipeng Xu, Marc Habermann, Michael Zollhoefer, Florian Bernard, Hyeongwoo Kim, Wenping Wang, Christian Theobalt
2021 arXiv   pre-print
Synthesizing realistic videos of humans using neural networks has been a popular alternative to the conventional graphics-based rendering pipeline due to its high efficiency.  ...  Given the pose information, the first CNN predicts a dynamic texture map that contains time-coherent high-frequency details, and the second CNN conditions the generation of the final video on the temporally  ...  ACKNOWLEDGMENT We thank our reviewers for their invaluable comments. We also thank Liqian Ma  ... 
arXiv:2001.04947v3 fatcat:ppii2ilexze7nkejshrohlky4u

NDF: Neural Deformable Fields for Dynamic Human Modelling [article]

Ruiqi Zhang, Jie Chen
2022 arXiv   pre-print
A neural network is then learned to map pose to the dynamics of NDF.  ...  We propose Neural Deformable Fields (NDF), a new representation for dynamic human digitization from a multi-view video.  ...  Neural Actor [14] learns an unposed implicit human model via inverse linear blend skinning functions (LBS).  ... 
arXiv:2207.09193v1 fatcat:pjmuxzhsdvg5lmuwlb3xlvhdb4

CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshes [article]

Kim Youwang, Kim Ji-Yeon, Tae-Hyun Oh
2022 arXiv   pre-print
We propose CLIP-Actor, a text-driven motion recommendation and neural mesh stylization system for human mesh animation.  ...  We demonstrate that CLIP-Actor produces plausible and human-recognizable style 3D human mesh in motion with detailed geometry and texture solely from a natural language prompt.  ...  The separate modeling of human and cloth meshes [6, 23, 32] , the neural extension of the parametric human mesh model [5, 12, 35] , the neural parametric approach [10, 11, 42] , and the neural implicit  ... 
arXiv:2206.04382v2 fatcat:txb5tg5fbjdp7jtebhb2cc4e6e

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars [article]

Tao Hu, Tao Yu, Zerong Zheng, He Zhang, Yebin Liu, Matthias Zwicker
2021 arXiv   pre-print
First, we learn to encode articulated human motions on a dense UV manifold of the human body surface.  ...  We propose a novel neural rendering pipeline, Hybrid Volumetric-Textural Rendering (HVTR), which synthesizes virtual human avatars from arbitrary poses efficiently and at high quality.  ...  Neural actor: Tang. Leap: Learning articulated occupancy of people. Neural free-view synthesis of human actors with pose con- ArXiv, abs/2104.06849, 2021. 2 trol.  ... 
arXiv:2112.10203v1 fatcat:fypyninknbfopb73m343dmeoua

DRaCoN – Differentiable Rasterization Conditioned Neural Radiance Fields for Articulated Avatars [article]

Amit Raj, Umar Iqbal, Koki Nagano, Sameh Khamis, Pavlo Molchanov, James Hays, Jan Kautz
2022 arXiv   pre-print
In this work, we present, DRaCoN, a framework for learning full-body volumetric avatars which exploits the advantages of both the 2D and 3D neural rendering techniques.  ...  volume), or 2D-based methods which learn photo-realistic renderings of avatars but lack accurate 3D representations.  ...  of 2D neural rendering for photorealism.  ... 
arXiv:2203.15798v1 fatcat:voygwvkn7rbh3pnsdghzv4447m

Neural Re-Rendering of Humans from a Single Image [article]

Kripasindhu Sarkar, Dushyant Mehta, Weipeng Xu, Vladislav Golyanik, Christian Theobalt
2021 arXiv   pre-print
To address these challenges, we propose a new method for neural re-rendering of a human under a novel user-defined pose and viewpoint, given one input image.  ...  Instead of a colour-based UV texture map, our approach further employs a learned high-dimensional UV feature map to encode appearance.  ...  Neural Re-Rendering of Humans from a Single Image  ... 
arXiv:2101.04104v1 fatcat:6snujhlbpbgd7nc4gcw62soivy

EgoRenderer: Rendering Human Avatars from Egocentric Camera Images [article]

Tao Hu, Kripasindhu Sarkar, Lingjie Liu, Matthias Zwicker, Christian Theobalt
2021 arXiv   pre-print
Our system renders photorealistic novel views of the actor and her motion from arbitrary virtual camera locations.  ...  We present EgoRenderer, a system for rendering full-body neural avatars of a person captured by a wearable, egocentric fisheye camera that is mounted on a cap or a VR headset.  ...  To handle the dynamic nature of the scene, we learn a person specific implicit texture map on a parametric model of humans.  ... 
arXiv:2111.12685v1 fatcat:ibpmcwfv2zclzmdvc7ivt26kfy

Deferred Neural Rendering: Image Synthesis using Neural Textures [article]

Justus Thies and Michael Zollhöfer and Matthias Nießner
2019 arXiv   pre-print
Specifically, we propose Neural Textures, which are learned feature maps that are trained as part of the scene capture process.  ...  This way, neural textures can be utilized to coherently re-render or manipulate existing video content in both static and dynamic environments at real-time rates.  ...  RELATED WORK Deferred Neural Rendering presents a new paradigm of image synthesis with learned neural textures and renderer.  ... 
arXiv:1904.12356v1 fatcat:r3vfiidi4fdgpnbr6phlvhsuuu

Dynamic Neural Garments [article]

Meng Zhang, Duygu Ceylan, Tuanfeng Wang, Niloy J. Mitra
2021 arXiv   pre-print
Technically, our solution generates a coarse garment proxy sequence, learns deep dynamic features attached to this template, and neurally renders the features to produce appearance changes such as folds  ...  Specifically, given the target joint motion sequence of an avatar, we propose dynamic neural garments to jointly simulate and render plausible dynamic garment appearance from an unseen viewpoint.  ...  Loss Function We train our dynamic neural rendering component to learn the parameters of the network and the neural texture jointly in an end-to-end manner.  ... 
arXiv:2102.11811v1 fatcat:sqtfd7d5ozaelil3mmp3kkol64

Real-time Deep Dynamic Characters [article]

Marc Habermann, Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Gerard Pons-Moll, Christian Theobalt
2021 arXiv   pre-print
We use a novel graph convolutional network architecture to enable motion-dependent deformation learning of body and clothing, including dynamics, and a neural generative dynamic texture model creates corresponding  ...  Our character model also features a learned dynamic texture model that accounts for photo-realistic motion-dependent appearance details, as well as view-dependent lighting effects.  ...  Monocular neural rendering approaches [Chan et al. 2019; Liu et al. 2020b Liu et al. , 2019b for humans learn a mapping from a CG rendering to a photo-realistic image, but their results have limited  ... 
arXiv:2105.01794v1 fatcat:q34njck5pffd3lryvzk2psxmoq

Learning to Paint With Model-Based Deep Reinforcement Learning

Zhewei Huang, Shuchang Zhou, Wen Heng
2019 2019 IEEE/CVF International Conference on Computer Vision (ICCV)  
By employing a neural renderer in model-based Deep Reinforcement Learning (DRL), our agents learn to determine the position and color of each stroke and make long-term plans to decompose texturerich images  ...  We show how to teach machines to paint like human painters, who can use a small number of strokes to create fantastic paintings.  ...  To achieve this, we set the difference of D scores from s t to s t+1 using equation (1) as the reward for guiding the learning of the actor.  ... 
doi:10.1109/iccv.2019.00880 dblp:conf/iccv/HuangZH19a fatcat:n6e3zylmzjdtflnta3buaxlhlq

Learning to Paint With Model-based Deep Reinforcement Learning [article]

Zhewei Huang, Wen Heng, Shuchang Zhou
2019 arXiv   pre-print
By employing a neural renderer in model-based Deep Reinforcement Learning (DRL), our agents learn to determine the position and color of each stroke and make long-term plans to decompose texture-rich images  ...  We show how to teach machines to paint like human painters, who can use a small number of strokes to create fantastic paintings.  ...  To achieve this, we set the difference of D scores from s t to s t+1 using equation (1) as the reward for guiding the learning of the actor.  ... 
arXiv:1903.04411v3 fatcat:j5dvpxwanzc4vda3aue3j4p6t4

NeuMan: Neural Human Radiance Field from a Single Video [article]

Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, Anurag Ranjan
2022 arXiv   pre-print
Photorealistic rendering and reposing of humans is important for enabling augmented reality experiences.  ...  Our method is able to learn subject specific details, including cloth wrinkles and accessories, from just a 10 seconds video clip, and to provide high quality renderings of the human under novel poses,  ...  Acknowledgement We thank Ashish Shrivastava, Russ Webb and Miguel Angel Bautista Martin for providing insightful review feedback.  ... 
arXiv:2203.12575v1 fatcat:d5rri2gfnrh2rl5u675sj6voje

UV Volumes for Real-time Rendering of Editable Free-view Human Performance [article]

Yue Chen, Xuan Wang, Xingyu Chen, Qi Zhang, Xiaoyu Li, Yu Guo, Jue Wang, Fei Wang
2022 arXiv   pre-print
Neural volume rendering enables photo-realistic renderings of a human performer in free-view, a critical task in immersive VR/AR applications.  ...  ., non-smooth) human appearance from the 3D volume, and encodes them into 2D neural texture stacks (NTS).  ...  However, learning a backward warp field is highly under-constrained, since the backward warp field is pose-dependent ]. Neural Actor ] takes the texture map as latent variables.  ... 
arXiv:2203.14402v3 fatcat:mlisaodfurck5gmjim22473elq

Dressing Avatars: Deep Photorealistic Appearance for Physically Simulated Clothing [article]

Donglai Xiang, Timur Bagautdinov, Tuur Stuyck, Fabian Prada, Javier Romero, Weipeng Xu, Shunsuke Saito, Jingfan Guo, Breannan Smith, Takaaki Shiratori, Yaser Sheikh, Jessica Hodgins (+1 others)
2022 arXiv   pre-print
To this end, we introduce pose-driven avatars with explicit modeling of clothing that exhibit both realistic clothing dynamics and photorealistic appearance learned from real-world data.  ...  Modeling photorealistic appearance, however, usually requires physically-based rendering which is too expensive for interactive applications.  ...  However, the high frequency dynamics of the skirt itself is hard for the network to learn, further jeopardizing the modeling of texture, as manifested by the blurry regions at the bottom of the skirt in  ... 
arXiv:2206.15470v1 fatcat:gmktehuxznbadga7qwdeko6ru4
« Previous Showing results 1 — 15 out of 1,860 results