22 Hits in 4.7 sec

Animatable Neural Radiance Fields from Monocular RGB Videos [article]

Jianchuan Chen, Ying Zhang, Di Kang, Xuefei Zhe, Linchao Bao, Xu Jia, Huchuan Lu
2021 arXiv   pre-print
We present animatable neural radiance fields (animatable NeRF) for detailed human avatar creation from monocular videos.  ...  Our approach extends neural radiance fields (NeRF) to the dynamic scenes with human movements via introducing explicit pose-guided deformation while learning the scene representation network.  ...  Objective Functions Given a monocular video sequence, we learn the animatable neural radiance field with the following objective function L = L c + L p + λ d * L d (7) where L c , L p , and L d are reconstruction  ... 
arXiv:2106.13629v2 fatcat:tg3f5jyyhbbcjhpisay5pe6b5e

DRaCoN – Differentiable Rasterization Conditioned Neural Radiance Fields for Articulated Avatars [article]

Amit Raj, Umar Iqbal, Koki Nagano, Sameh Khamis, Pavlo Molchanov, James Hays, Jan Kautz
2022 arXiv   pre-print
While DiffRas helps in obtaining photo-realistic image quality, c-NeRF, which employs signed distance fields (SDF) for 3D representations, helps to obtain fine 3D geometric details.  ...  In this work, we present, DRaCoN, a framework for learning full-body volumetric avatars which exploits the advantages of both the 2D and 3D neural rendering techniques.  ...  In this work, we propose Differentiable Rasterization Conditioned Neural Radiance Field (DRaCoN), that leverages the advantages of both 2D and 3D neural rendering methods.  ... 
arXiv:2203.15798v1 fatcat:voygwvkn7rbh3pnsdghzv4447m

Animatable Neural Implicit Surfaces for Creating Avatars from Videos [article]

Sida Peng, Shangzhan Zhang, Zhen Xu, Chen Geng, Boyi Jiang, Hujun Bao, Xiaowei Zhou
2022 arXiv   pre-print
This paper aims to reconstruct an animatable human model from a video of very sparse camera views.  ...  Some recent works represent human geometry and appearance with neural radiance fields and utilize parametric human models to produce deformation fields for animation, which enables them to recover detailed  ...  Animatable NeRF [56] represents the human model as a neural radiance field (NeRF) [45] in the canonical space.  ... 
arXiv:2203.08133v1 fatcat:dv7vpk44m5anlbi4aprc3kdqca

I M Avatar: Implicit Morphable Head Avatars from Videos [article]

Yufeng Zheng, Victoria Fernández Abrevaya, Marcel C. Bühler, Xu Chen, Michael J. Black, Otmar Hilliges
2022 arXiv   pre-print
To tackle this problem, we propose IMavatar (Implicit Morphable avatar), a novel method for learning implicit head avatars from monocular videos.  ...  Neural volumetric representations approach photorealism but are hard to animate and do not generalize well to unseen expressions.  ...  The closest work to ours is NerFACE [22] , which recovers an animatable neural radiance field from a single monocular video by conditioning on the expression parameters from a 3DMM.  ... 
arXiv:2112.07471v5 fatcat:px4mx4itbrhsvfhrkzcgfkhuue

BANMo: Building Animatable 3D Neural Models from Many Casual Videos [article]

Gengshan Yang, Minh Vo, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, Hanbyul Joo
2021 arXiv   pre-print
BANMo builds high-fidelity, articulated 3D models (including shape and animatable skinning weights) from many monocular casual videos in a differentiable rendering framework.  ...  Our key insight is to merge three schools of thought; (1) classic deformable shape models that make use of articulated bones and blend skinning, (2) volumetric neural radiance fields (NeRFs) that are amenable  ...  In ICCV, rigid neural radiance fields: Reconstruction and novel view 2019. 2 synthesis of a dynamic scene from monocular video.  ... 
arXiv:2112.12761v2 fatcat:creiz2vswzdozoghhury7g5aza

NeuMan: Neural Human Radiance Field from a Single Video [article]

Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, Anurag Ranjan
2022 arXiv   pre-print
Those rough geometry estimates allow us to create a warping field from the observation space to the canonical pose-independent space, where we train the human model in.  ...  from novel views, together with the background.  ...  Preliminary: Neural Radiance Fields (NeRF) For completeness, we quickly review the standard NeRF model [27] .  ... 
arXiv:2203.12575v1 fatcat:d5rri2gfnrh2rl5u675sj6voje

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars [article]

Tao Hu, Tao Yu, Zerong Zheng, He Zhang, Yebin Liu, Matthias Zwicker
2021 arXiv   pre-print
radiance field.  ...  Hence we employ only a rough volumetric representation using a pose-conditioned downsampled neural radiance field (PD-NeRF), which we can render efficiently at low resolutions.  ...  Animatable neural radiance fields NeuralBody 0.212 155.838 18.20 - from monocular rgb video.  ... 
arXiv:2112.10203v1 fatcat:fypyninknbfopb73m343dmeoua

MoFaNeRF: Morphable Facial Neural Radiance Field [article]

Yiyu Zhuang, Hao Zhu, Xusen Sun, Xun Cao
2021 arXiv   pre-print
We propose a parametric model that maps free-view images into a vector space of coded facial shape, expression and appearance using a neural radiance field, namely Morphable Facial NeRF.  ...  To the best of our knowledge, our work is the first facial parametric model built upon a neural radiance field that can be used in fitting, generation and manipulation.  ...  Animatable neural radiance fields from monocular rgb videos. arXiv preprint not raise infringement issues We have signed the license  ... 
arXiv:2112.02308v1 fatcat:xvd6vh3lqjghjg75nd6gomccke

UV Volumes for Real-time Rendering of Editable Free-view Human Performance [article]

Yue Chen, Xuan Wang, Xingyu Chen, Qi Zhang, Xiaoyu Li, Yu Guo, Jue Wang, Fei Wang
2022 arXiv   pre-print
It separates the high-frequency (i.e., non-smooth) human appearance from the 3D volume, and encodes them into 2D neural texture stacks (NTS).  ...  Neural volume rendering enables photo-realistic renderings of a human performer in free-view, a critical task in immersive VR/AR applications.  ...  radiance field; 3) AN: Animatable-NeRF, which uses neural blend weight fields to generate correspondences between observation and canonical space.  ... 
arXiv:2203.14402v3 fatcat:mlisaodfurck5gmjim22473elq

Human View Synthesis using a Single Sparse RGB-D Input [article]

Phong Nguyen, Nikolaos Sarafianos, Christoph Lassner, Janne Heikkila, Tony Tung
2021 arXiv   pre-print
Aiming to address these limitations, we present a novel view synthesis framework to generate realistic renders from unseen views of any human captured from a single-view sensor with sparse RGB-D, similar  ...  We show our method generates high-quality novel views of synthetic and real human actors given a single sparse RGB-D input.  ...  Neural articulated radiance field.  ... 
arXiv:2112.13889v2 fatcat:u6e2uuinxra2lnrknjmrcsd6yq

Neural Head Avatars from Monocular RGB Videos [article]

Philip-William Grassal
2022 arXiv   pre-print
Our representation can be learned from a monocular RGB portrait video that features a range of different expressions and views.  ...  We present Neural Head Avatars, a novel neural representation that explicitly models the surface geometry and appearance of an animatable human avatar that can be used for teleconferencing in AR/VR or  ...  Acknowledgements This project has received funding from the DFG in the joint German-Japan-France grant agreement (RO 4804/3-1), the ERC Starting Grant Scan2CAD (804724), and a Microsoft Research Grant.  ... 
arXiv:2112.01554v2 fatcat:svlbhpdpkvh65a3vshy6kqjeei

Advances in Neural Rendering [article]

Ayush Tewari, Justus Thies, Ben Mildenhall, Pratul Srinivasan, Edgar Tretschk, Yifan Wang, Christoph Lassner, Vincent Sitzmann, Ricardo Martin-Brualla, Stephen Lombardi, Tomas Simon, Christian Theobalt (+5 others)
2022 arXiv   pre-print
., from a depth sensor), volumetric grids (e.g., from a CT scan), or implicit surface functions (e.g., truncated signed distance fields).  ...  Neural rendering is closely related, and combines ideas from classical computer graphics and machine learning to create algorithms for synthesizing images from real-world observations.  ...  While A-NeRF is trained on monocular videos, Animatable Neural Radiance Fields (ANRF) [PDW * 21] is a skeletondriven approach for human model reconstruction from multi-view videos.  ... 
arXiv:2111.05849v2 fatcat:nbvkfg2bjvgqdopdqwl33rt4ii

LISA: Learning Implicit Shape and Appearance of Hands [article]

Enric Corona, Tomas Hodan, Minh Vo, Francesc Moreno-Noguer, Chris Sweeney, Richard Newcombe, Lingni Ma
2022 arXiv   pre-print
We experimentally demonstrate that LISA can accurately reconstruct a dynamic hand from monocular or multi-view sequences, achieving a noticeably higher quality of reconstructed hand shapes compared to  ...  This paper proposes a do-it-all neural model of human hands, named LISA.  ...  Similar to radiance field definition, this is defined as: ψ : (x, d) → (s, c) , (5) which returns the SDF value s ∈ R and the color value c for a query 3D point x and the view direction d.  ... 
arXiv:2204.01695v1 fatcat:53v7sunsjze6ndb5k2exeqz6x4

Real-Time Neural Character Rendering with Pose-Guided Multiplane Images [article]

Hao Ouyang, Bo Zhang, Pan Zhang, Hao Yang, Jiaolong Yang, Dong Chen, Qifeng Chen, Fang Wen
2022 arXiv   pre-print
We propose pose-guided multiplane image (MPI) synthesis which can render an animatable character in real scenes with photorealistic quality.  ...  Our method achieves such expressive and animatable character rendering all in real time, serving as a promising solution for practical applications.  ...  One recent notable work is to model the scene as a neural radiance field (NeRF) whose color and volumetric density across the continuous coordinates are parameterized by the multilayer perceptron (MLP)  ... 
arXiv:2204.11820v1 fatcat:svrgg7sr2vgdfpua74rgo6cqge

Self-supervised Neural Articulated Shape and Appearance Models [article]

Fangyin Wei, Rohan Chabra, Lingni Ma, Christoph Lassner, Michael Zollhöfer, Szymon Rusinkiewicz, Chris Sweeney, Richard Newcombe, Mira Slavcheva
2022 arXiv   pre-print
Compared to state of the art that uses direct 3D supervision and does not output appearance, we recover more faithful geometry and appearance from 2D observations only.  ...  D-NeRF [49] approximates the radiance field of a deforming shape by using time as an input to the system but only works for a single scene.  ...  Neural Representations for Appearance. Neural Radiance Fields (NeRF) model the appearance of static scenes via a coordinate-based scene representation [36] .  ... 
arXiv:2205.08525v1 fatcat:fjmtfkjn7jbkzafuncxzsu5lje
« Previous Showing results 1 — 15 out of 22 results