105 Hits in 6.1 sec

Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images [article]

Sai Bi, Zexiang Xu, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoffroy, David Kriegman, Ravi Ramamoorthi
2020 arXiv   pre-print
At the heart of Deep Reflectance Volumes is a novel volumetric scene representation consisting of opacity, surface normal and reflectance voxel grids.  ...  We present a deep learning approach to reconstruct scene appearance from unstructured images captured under collocated point lighting.  ...  We thank Giljoo Nam for help with the comparisons. This work was supported in part by ONR grants N000141712687, N000141912293, N000142012529, NSF grant 1617234, Adobe, the Ronald L.  ... 
arXiv:2007.09892v1 fatcat:6h642u5ktnel5nzp7gkozgvyae

Deep Neural Models for Illumination Estimation and Relighting: A Survey

Farshad Einabadi, Jean‐Yves Guillemaut, Adrian Hilton
2021 Computer graphics forum (Print)  
Recently, the application of deep neural computing to illumination estimation, relighting and inverse rendering has shown promising results.  ...  We examine in detail the attributes of the proposed approaches, presented in three categories: scene illumination estimation, relighting with reflectance-aware scenespecific representations and finally  ...  [GLD*19b] combine common approaches for volumetric performance capture of humans, for example [PKC*17] with an image-based rendering from a light stage setup to overcome the issue of relightability  ... 
doi:10.1111/cgf.14283 fatcat:iaidifr5m5gwvmg3exbzvyrrem

Generative Modelling of BRDF Textures from Flash Images [article]

Philipp Henzler, Valentin Deschaintre, Niloy J. Mitra, Tobias Ritschel
2021 arXiv   pre-print
A user study compares our approach favorably to previous work, even those with access to BRDF supervision.  ...  We learn a latent space for easy capture, consistent interpolation, and efficient reproduction of visual material appearance.  ...  We thank the authors of Aittala et al. [2016] , Gao et al. [2019] , Guo et al. [2020b] , for providing their implementations and helping with the comparisons.  ... 
arXiv:2102.11861v2 fatcat:6me7cyt4jbhivbvpou3irpfuwi

Deferred Neural Rendering: Image Synthesis using Neural Textures [article]

Justus Thies and Michael Zollhöfer and Matthias Nießner
2019 arXiv   pre-print
Specifically, we propose Neural Textures, which are learned feature maps that are trained as part of the scene capture process.  ...  Both neural textures and deferred neural renderer are trained end-to-end, enabling us to synthesize photo-realistic images even when the original 3D content was imperfect.  ...  of our Neural Textures together with the Deferred Neural Renderer.  ... 
arXiv:1904.12356v1 fatcat:r3vfiidi4fdgpnbr6phlvhsuuu

TexMesh: Reconstructing Detailed Human Texture and Geometry from RGB-D Video [article]

Tiancheng Zhi, Christoph Lassner, Tony Tung, Carsten Stoll, Srinivasa G. Narasimhan, Minh Vo
2020 arXiv   pre-print
We present TexMesh, a novel approach to reconstruct detailed human meshes with high-resolution full-body texture from RGB-D video. TexMesh enables high quality free-viewpoint rendering of humans.  ...  Given the RGB frames, the captured environment map, and the coarse per-frame human mesh from RGB-D tracking, our method reconstructs spatiotemporally consistent and detailed per-frame meshes along with  ...  Acknowledgements We thank Junbang Liang, Yinghao Huang, and Nikolaos Sarafianos for their help with the synthetic training data generation.  ... 
arXiv:2008.00158v3 fatcat:i7icmrik3zdpjolo5lxjxubh2m

A Neural Rendering Framework for Free-Viewpoint Relighting [article]

Zhang Chen, Anpei Chen, Guli Zhang, Chengyuan Wang, Yu Ji, Kiriakos N. Kutulakos, Jingyi Yu
2020 arXiv   pre-print
We present a novel Relightable Neural Renderer (RNR) for simultaneous view synthesis and relighting using multi-view image inputs.  ...  Existing neural rendering (NR) does not explicitly model the physical rendering process and hence has limited capabilities on relighting.  ...  Approaches such as neural point cloud [2] , neural volume [63] and neural texture [69] utilize deep representations for 3D content.  ... 
arXiv:1911.11530v2 fatcat:s5lv6nmdxja2jncm5kv4ysasde

State of the Art on Neural Rendering [article]

Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, Rohit Pandey, Sean Fanello (+7 others)
2020 arXiv   pre-print
Starting with an overview of the underlying computer graphics and machine learning concepts, we discuss critical aspects of neural rendering approaches.  ...  Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering  ...  Free Viewpoint Videos Free Viewpoint Videos, also known as Volumetric Performance Capture, rely on multi-camera setups to acquire the 3D shape and texture of performers.  ... 
arXiv:2004.03805v1 fatcat:6qs7ddftkfbotdlfd4ks7llovq

Neural Light Transport for Relighting and View Synthesis [article]

Xiuming Zhang, Sean Fanello, Yun-Ta Tsai, Tiancheng Sun, Tianfan Xue, Rohit Pandey, Sergio Orts-Escolano, Philip Davidson, Christoph Rhemann, Paul Debevec, Jonathan T. Barron, Ravi Ramamoorthi (+1 others)
2020 arXiv   pre-print
With this learned LT, one can relight the scene photorealistically with a directional light or an HDRI map, synthesize novel views with view-dependent effects, or do both simultaneously, all in a unified  ...  added to a physically-accurate diffuse base rendering.  ...  Assuming accurate geometry is available, one can render Gbu ers in the image space and then apply Deep Shading [Nalbach et al. 2017 ], a convolutional neural network, to perform the nal rendering.  ... 
arXiv:2008.03806v2 fatcat:ifhs7lpufne2pf5twp62w2trja

NeLF: Neural Light-transport Field for Portrait View Synthesis and Relighting [article]

Tiancheng Sun, Kai-En Lin, Sai Bi, Zexiang Xu, Ravi Ramamoorthi
2021 arXiv   pre-print
To this end, we present a system for portrait view synthesis and relighting: given multiple portraits, we use a neural network to predict the light-transport field in 3D space, and from the predicted Neural  ...  Our method achieves simultaneous view synthesis and relighting given multi-view portraits as the input, and achieves state-of-the-art results.  ...  Thanks to the reviewers for the valuable feedback, and to the anonymous volunteers for being captured.  ... 
arXiv:2107.12351v1 fatcat:c3riri2ujvcmlayhrvpsx52ybi

Neural Ray-Tracing: Learning Surfaces and Reflectance for Relighting and View Synthesis [article]

Julian Knodt, Joe Bartusek, Seung-Hwan Baek, Felix Heide
2021 arXiv   pre-print
Recent neural rendering methods have demonstrated accurate view interpolation by predicting volumetric density and color with a neural network.  ...  We validate the proposed approach for scene editing, relighting and reflectance estimation learned from synthetic and captured views on a subset of NeRV's datasets.  ...  Recently, attempts to incorporate these textures and relighting into existing neural pipelines have been proposed [1, 2, 28] .  ... 
arXiv:2104.13562v2 fatcat:bgy25vm77jaudfcx4l4ftvwvfq

Appearance Editing with Free-viewpoint Neural Rendering [article]

Pulkit Gera, Aakash KT, Dhawal Sirikonda, Parikshit Sakurikar, P.J. Narayanan
2021 arXiv   pre-print
We present a neural rendering framework for simultaneous view synthesis and appearance editing of a scene from multi-view images captured under known environment illumination.  ...  Existing approaches either achieve view synthesis alone or view synthesis along with relighting, without direct control over the scene's appearance.  ...  Deferred Neural Lighting (DNL) [8] extended this to allow relighting by projecting learnt neural textures onto a rough proxy geometry using a scene-dependent neural rendering network for relighting.  ... 
arXiv:2110.07674v1 fatcat:k5ewg5vfovfsjp6cqlpv7sbknq

NeRD: Neural Reflectance Decomposition from Image Collections [article]

Mark Boss, Raphael Braun, Varun Jampani, Jonathan T. Barron, Ce Liu, Hendrik P.A. Lensch
2021 arXiv   pre-print
In addition, we also propose techniques to convert the learned reflectance volume into a relightable textured mesh enabling fast real-time rendering with novel illuminations.  ...  We propose a neural reflectance decomposition (NeRD) technique that uses physically-based rendering to decompose the scene into spatially varying BRDF material properties.  ...  Two scenes are highlighted to show the decomposition performance of our method. Notice the accurate performance in relighting with unseen illuminations.  ... 
arXiv:2012.03918v4 fatcat:mueqzzmffncftkd2d73s5fwnya

Extracting Triangular 3D Models, Materials, and Lighting From Images [article]

Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Müller, Sanja Fidler
2022 arXiv   pre-print
We leverage recent work in differentiable rendering, coordinate-based networks to compactly represent volumetric texturing, alongside differentiable marching tetrahedrons to enable gradient-based optimization  ...  Unlike recent multi-view reconstruction approaches, which typically produce entangled 3D representations encoded in neural networks, we output triangle meshes with spatially-varying materials and environment  ...  We leverage Deep Marching Tetrahedra [60] (DMTet) in a 2D supervision setting through differentiable rendering.  ... 
arXiv:2111.12503v3 fatcat:pnd3yiexwvcqlkgxbebcwadlki

Free-viewpoint Indoor Neural Relighting from Multi-view Stereo [article]

Julien Philip and Sébastien Morgenthaler and Michaël Gharbi and George Drettakis
2021 arXiv   pre-print
We introduce a neural relighting algorithm for captured indoors scenes, that allows interactive free-viewpoint navigation.  ...  We show results of our algorithm relighting real indoor scenes and performing free-viewpoint navigation with complex and realistic glossy reflections, which so far remained out of reach for view-synthesis  ...  Another neural representation, deep neural textures [Thies et al. 2019 ] allows view synthesis for glossy objects at the cost of very dense capture. use them to relight single objects.  ... 
arXiv:2106.13299v1 fatcat:iizcgmc7gjgidca4lftcnan62q

Neural Radiance Fields for Outdoor Scene Relighting [article]

Viktor Rudnev and Mohamed Elgharib and William Smith and Lingjie Liu and Vladislav Golyanik and Christian Theobalt
2021 arXiv   pre-print
., the first approach for outdoor scene relighting based on neural radiance fields.  ...  For each timing, a 360 degrees environment map is captured together with a colour-calibration chequerboard to allow accurate numerical evaluations on real data against ground truth.  ...  Deep relightable tex- Recognition (CVPR), 2019. 2 tures - volumetric performance capture with neural render- [6] Kaiwen Guo  ... 
arXiv:2112.05140v1 fatcat:kgsqa4x73rcbziejajqkzrajqe
« Previous Showing results 1 — 15 out of 105 results