128,196 Hits in 5.2 sec

Differentiable Surface Rendering via Non-Differentiable Sampling [article]

Forrester Cole, Kyle Genova, Avneesh Sud, Daniel Vlasic, Zhoutong Zhang
2021 arXiv   pre-print
The method first samples the surface using non-differentiable rasterization, then applies differentiable, depth-aware point splatting to produce the final image.  ...  In particular, we show for the first time efficient, differentiable rendering of an isosurface extracted from a neural radiance field (NeRF), and demonstrate surface-based, rather than volume-based, rendering  ...  Our method applies to any surface that can be expressed as a non-differentiable sampling and a differentiable evaluation function.  ... 
arXiv:2108.04886v1 fatcat:ko5d6jnwyjahzlq3sjdn2ljtgi

Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data [article]

Sebastian Lunz, Yingzhen Li, Andrew Fitzgibbon, Nate Kushman
2020 arXiv   pre-print
To account for the non-differentiability, we introduce a proxy neural renderer to match the output of the non-differentiable renderer.  ...  this non-differentiable process in various ways.  ...  In particular two aspects of the rendering process are non-differentiable: (1) the rasterization step inside of the renderer is inherently non-differentiable as a result of occlusion and (2) sampling the  ... 
arXiv:2002.12674v1 fatcat:jplsyd2sh5ggniv4fv45lfkkyi

Reparameterizing discontinuous integrands for differentiable rendering

Guillaume Loubet, Nicolas Holzschuch, Wenzel Jakob
2019 ACM Transactions on Graphics  
We show that our method only requires a few samples to produce gradients with low bias and variance for challenging cases such as glossy reflections and shadows.  ...  Finally, we use our differentiable path tracer to reconstruct the 3D geometry and materials of several real-world objects from a set of reference photographs.  ...  integrands non-differentiable.  ... 
doi:10.1145/3355089.3356510 fatcat:pouh7is55ndkhnz6grjtz7ixse

SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization [article]

Yue Jiang, Dantong Ji, Zhizhong Han, Matthias Zwicker
2022 arXiv   pre-print
We propose SDFDiff, a novel approach for image-based shape optimization using differentiable rendering of 3D shapes represented by signed distance functions (SDFs).  ...  We further demonstrate that our SDF-based differentiable renderer can be integrated with deep learning models, which opens up options for learning approaches on 3D objects without 3D supervision.  ...  . tion approach using shape optimization via differentiable rendering.  ... 
arXiv:1912.07109v2 fatcat:7gjbfiqpnfeb3bt2btskwu6p6a

MeshSDF: Differentiable Iso-Surface Extraction [article]

Edoardo Remelli, Artem Lukoianov, Stephan R. Richter, Benoît Guillard, Timur Bagautdinov, Pierre Baque, Pascal Fua
2020 arXiv   pre-print
We use two different applications to validate our theoretical insight: Single-View Reconstruction via Differentiable Rendering and Physically-Driven Shape Optimization.  ...  Our key insight is that by reasoning on how implicit field perturbations impact local surface geometry, one can ultimately differentiate the 3D location of surface samples with respect to the underlying  ...  Accelerating Iso-Surface Extraction Recall that our approach to iso-surface differentiation method is independent from the technique used to extract surface samples, meaning that any non-differentiable  ... 
arXiv:2006.03997v2 fatcat:msmzxi66xrbjtjym3lylz7zlmm

PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting [article]

Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, Noah Snavely
2021 arXiv   pre-print
We present PhySG, an end-to-end inverse rendering pipeline that includes a fully differentiable renderer and can reconstruct geometry, materials, and illumination from scratch from a set of RGB input images  ...  The use of spherical Gaussians allows us to efficiently solve for approximate light transport, and our method works on scenes with challenging non-Lambertian reflectance captured under natural, static  ...  Back-propagating through the surface normal n is straightforward via auto-differentiation [36] .  ... 
arXiv:2104.00674v1 fatcat:jcfzs55rsba5vjys2daudlcanq

DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer [article]

Wenzheng Chen and Joey Litalien and Jun Gao and Zian Wang and Clement Fuji Tsang and Sameh Khamis and Or Litany and Sanja Fidler
2021 arXiv   pre-print
We consider the challenging problem of predicting intrinsic object properties from a single image by exploiting differentiable renderers.  ...  Many previous learning-based approaches for inverse graphics adopt rasterization-based renderers and assume naive lighting and material models, which often fail to account for non-Lambertian, specular  ...  Indeed, when β is small enough (β → 0) and we deal with a highly non-Lambertian surface, a small number of MC samples are enough to estimate direct illumination, which in turn implies faster render speed  ... 
arXiv:2111.00140v1 fatcat:bdlbtkktm5hhha33mcnxv7bbh4

Terahertz Reconstructive Imaging: A novel technique to differentiate healthy and diseased human skin

Anis Rahman
2019 British Journal of Cancer Research  
Here, we imaged 21 ex-vivo skin samples using a continuous wave terahertz reflectance reconstructive imaging technique in which the images were rendered by a computer algorithm.  ...  That, along with further optimization of the imaging parameters, can bring this modality to the bedside for effective non-invasive diagnosis.  ...  The outer surface of the sample was oriented to face the beam.  ... 
doi:10.31488/bjcr.121 fatcat:y3k5blffifbtncl3p5gt6ao64i

Differentiable Rendering: A Survey [article]

Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, Adrien Gaidon
2020 arXiv   pre-print
Differentiable rendering is a novel field which allows the gradients of 3D objects to be calculated and propagated through images.  ...  This paper reviews existing literature and discusses the current state of differentiable rendering, its applications and open research problems.  ...  Non-Differentiable Rendering Several non-differentiable rasterizer-based and ray tracingbased rendering libraries have been developed over the last few decades.  ... 
arXiv:2006.12057v2 fatcat:6zj6besdcnebrb4qww4u4jusji

Differentiable Stereopsis: Meshes from multiple views using differentiable rendering [article]

Shubham Goel, Georgia Gkioxari, Jitendra Malik
2021 arXiv   pre-print
We pair traditional stereopsis and modern differentiable rendering to build an end-to-end model which predicts textured 3D meshes of objects with varying topologies and shape.  ...  We frame stereopsis as an optimization problem and simultaneously update shape and cameras via simple gradient descent.  ...  Texture Rendering We described how to sample texture for a point x on the mesh surface.  ... 
arXiv:2110.05472v1 fatcat:zinloboebvhqdk76pkdgxpxwx4

DeepMesh: Differentiable Iso-Surface Extraction [article]

Benoit Guillard, Edoardo Remelli, Artem Lukoianov, Stephan R. Richter, Timur Bagautdinov, Pierre Baque, Pascal Fua
2022 arXiv   pre-print
We validate our theoretical insight through several applications: Single view 3D Reconstruction via Differentiable Rendering, Physically-Driven Shape Optimization, Full Scene 3D Reconstruction from Scans  ...  Our key insight is that by reasoning on how implicit field perturbations impact local surface geometry, one can ultimately differentiate the 3D location of surface samples with respect to the underlying  ...  This lets us extract the explicit surface using a non-differentiable algorithm, such as Marching Cubes, and then perform the backward pass through the extracted surface samples.  ... 
arXiv:2106.11795v2 fatcat:lwuetyfe5nfo3fqtrwcy3qll5a

Differentiable Direct Volume Rendering [article]

Sebastian Weiss, Rüdiger Westermann
2021 arXiv   pre-print
We present a differentiable volume rendering solution that provides differentiability of all continuous parameters of the volume rendering process.  ...  We have tailored the approach to volume rendering by enforcing a constant memory footprint via analytic inversion of the blending functions.  ...  While a number of approaches have been proposed for differentiable surface rendering [20] , approaches focusing on differentiable rendering in the context of volume visualization are rare.  ... 
arXiv:2107.12672v1 fatcat:f5kv6vwanfb6dfkw6e3ql2xrhu

UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction [article]

Michael Oechsle, Songyou Peng, Andreas Geiger
2021 arXiv   pre-print
Our key insight is that implicit surface models and radiance fields can be formulated in a unified way, enabling both surface and volume rendering using the same model.  ...  This unified perspective enables novel, more efficient sampling procedures and the ability to reconstruct accurate surfaces without input masks.  ...  As the surface can be estimated directly from the occupancy field o θ via root-finding [38] , this eliminates the need for hierarchical two-stage sampling as in NeRF.  ... 
arXiv:2104.10078v2 fatcat:lp6qk36furafnj3tp2thrzoqty

Accelerating 3D Deep Learning with PyTorch3D [article]

Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, Georgia Gkioxari
2020 arXiv   pre-print
It includes a fast, modular differentiable renderer for meshes and point clouds, enabling analysis-by-synthesis approaches.  ...  Compared with other differentiable renderers, PyTorch3D is more modular and efficient, allowing users to more easily extend it while also gracefully scaling to large meshes and images.  ...  We form point clouds by sampling uniformly from mesh surfaces.  ... 
arXiv:2007.08501v1 fatcat:ynk2ntc5bncjlc3bc7n6k3geia

Coupling Explicit and Implicit Surface Representations for Generative 3D Modeling [article]

Omid Poursaeed and Matthew Fisher and Noam Aigerman and Vladimir G. Kim
2020 arXiv   pre-print
We propose a novel neural architecture for representing 3D surfaces, which harnesses two complementary shape representations: (i) an explicit representation via an atlas, i.e., embeddings of 2D domains  ...  This process is computationally efficient, and can be directly used by differentiable rasterizers, enabling training our hybrid representation with image-based losses.  ...  Such a loss can be added via differentiable layers that either extract the level set of the OccupancyNet or convert AtlasNet surface to an implicit function.  ... 
arXiv:2007.10294v2 fatcat:pl7yk2ucpnfonifedy5v7nediq
« Previous Showing results 1 — 15 out of 128,196 results