300 Hits in 4.3 sec

Self-calibrating Photometric Stereo by Neural Inverse Rendering [article]

Junxuan Li, Hongdong Li
2022 arXiv   pre-print
The specularities are used explicitly to solve uncalibrated photometric stereo via a neural inverse rendering process.  ...  Our method leverages a physically based rendering equation by minimizing the reconstruction error on a per-object-basis.  ...  Acknowledgments This research is funded in part by ARC-Discovery grants (DP190102261 and DP220100800), a gift from Baidu RAL, as well as a Ford Alliance grant to Hongdong Li.  ... 
arXiv:2207.07815v1 fatcat:evi7utx2frar3dek42darxxpka

Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces [article]

Berk Kaya, Suryansh Kumar, Carlos Oliveira, Vittorio Ferrari, Luc Van Gool
2021 arXiv   pre-print
To bypass this difficulty, we propose an uncalibrated neural inverse rendering approach to this problem.  ...  This paper presents an uncalibrated deep neural network framework for the photometric stereo problem.  ...  This work was funded by Focused Research Award from Google (CVL, ETH 2019-HE-318, 2019-HE-323). We thank Vincent Vanweddingen from KU Lueven for providing some datasets for our experiments.  ... 
arXiv:2012.06777v3 fatcat:mfmt5jigxbewtluvmlo3qkdydy

DeepPS2: Revisiting Photometric Stereo Using Two Differently Illuminated Images [article]

Ashish Tiwari, Shanmuganathan Raman
2022 arXiv   pre-print
We propose an inverse rendering-based deep learning framework, called DeepPS2, that jointly performs surface normal, albedo, lighting estimation, and image relighting in a completely self-supervised manner  ...  In this work, we attempt to address an under-explored problem of photometric stereo using just two differently illuminated images, referred to as the PS2 problem.  ...  The photometric stereo problem using deep neural networks has been addressed either under a calibrated (known lightings) or an uncalibrated (unknown lightings) setting.  ... 
arXiv:2207.02025v2 fatcat:kcbujlxvq5dyjc45v6hjx3ab6a

Deep Uncalibrated Photometric Stereo via Inter-Intra Image Feature Fusion [article]

Fangzhou Gao, Meng Wang, Lianghao Zhang, Li Wang, Jiawan Zhang
2022 arXiv   pre-print
Previous methods use optimization-based neural inverse rendering or a single size-independent pooling layer to deal with multiple inputs, which are inefficient for utilizing information among input images  ...  Uncalibrated photometric stereo is proposed to estimate the detailed surface normal from images under varying and unknown lightings.  ...  Some researchers utilized optimization-based neural inverse rendering to solve this problem [22] . Kaya et al. [22] optimized the surface normal by the neural rendering layers.  ... 
arXiv:2208.03440v1 fatcat:uj57jcxdcfhljfjrrt4pcjhcwy

Universal Photometric Stereo Network using Global Lighting Contexts [article]

Satoshi Ikehata
2022 arXiv   pre-print
We use them like lighting parameters in a calibrated photometric stereo network to recover surface normal vectors pixelwisely.  ...  This paper tackles a new photometric stereo task, named universal photometric stereo.  ...  Aknowledgement This work was supported by JSPS KAKENHI Grant Number JP22K17919.  ... 
arXiv:2206.02452v1 fatcat:whpkkkxiv5gu3cbagm4hxiqngu

RC-MVSNet: Unsupervised Multi-View Stereo with Neural Rendering [article]

Di Chang, Aljaž Božič, Tong Zhang, Qingsong Yan, Yingcong Chen, Sabine Süsstrunk, Matthias Nießner
2022 arXiv   pre-print
In this work, we propose a novel approach with neural rendering (RC-MVSNet) to solve such ambiguity issues of correspondences among views.  ...  Existing methods are built upon the assumption that corresponding pixels share similar photometric features.  ...  . b) Rendering Consistency Network generates image and depth by neural rendering under the guidance of depth priors. c) The rendered image is supervised by the reference view synthesis loss. d) The rendered  ... 
arXiv:2203.03949v4 fatcat:cicmdakmcbbnnoh6lnu6nyn6wq

Deep Photometric Stereo Network with Multi-Scale Feature Aggregation

Chanki Yu, Sang Wook Lee
2020 Sensors  
Our experiments were performed with a DiLiGent photometric stereo benchmark dataset consisting of ten real objects, and they demonstrated that the accuracies of our calibrated and uncalibrated photometric  ...  We present photometric stereo algorithms robust to non-Lambertian reflection, which are based on a convolutional neural network in which surface normals of objects with complex geometry and surface reflectance  ...  Taniai and Maehara presented an inverse rendering-based CNN architecture for a photometric stereo in an unsupervised manner.  ... 
doi:10.3390/s20216261 pmid:33153006 fatcat:h6w53ohfuraqbcs6xlcmigec2m

A CNN Based Approach for the Near-Field Photometric Stereo Problem [article]

Fotios Logothetis, Ignas Budvytis, Roberto Mecca, Roberto Cipolla
2020 arXiv   pre-print
We leverage recent improvements of deep neural networks for far-field Photometric Stereo and adapt them to near field setup.  ...  In this work, we propose the first CNN based approach capable of handling these realistic assumptions in Photometric Stereo.  ...  The first column shows the average Photometric Stereo image.  ... 
arXiv:2009.05792v1 fatcat:5n6vjyytxjah5jsctlgesu6aoa

Neural Reflectance for Shape Recovery with Shadow Handling [article]

Junxuan Li, Hongdong Li
2022 arXiv   pre-print
Tests on real-world images demonstrate that our method outperform existing methods by a significant margin.  ...  Our framework is entirely self-supervised, in the sense that it requires neither ground truth shape nor BRDF.  ...  Acknowledgments This research is funded in part by ARC-Discovery grants (DP 190102261 and DP220100800), a gift from Baidu RAL, as well as a Ford Alliance grant to Hongdong Li.  ... 
arXiv:2203.12909v1 fatcat:wrdx3gcvw5g5ngvvh4qhlwoesa

Edge-preserving Near-light Photometric Stereo with Neural Surfaces [article]

Heng Guo, Hiroaki Santo, Boxin Shi, Yasuyuki Matsushita
2022 arXiv   pre-print
This paper presents a near-light photometric stereo method that faithfully preserves sharp depth edges in the 3D reconstruction.  ...  stereo for avoiding differentiation errors at sharp depth edges, where the depth is represented as a neural function of the image coordinates.  ...  in recent multi-view inverse rendering techniques [36, 37] .  ... 
arXiv:2207.04622v1 fatcat:tajl3tftxnd5fkeuiemoxnfwha

CNN-PS: CNN-based Photometric Stereo for General Non-Convex Surfaces [article]

Satoshi Ikehata
2018 arXiv   pre-print
For training the network, we create a synthetic photometric stereo dataset that is generated by a physics-based renderer, therefore the global light transport is considered.  ...  Most conventional photometric stereo algorithms inversely solve a BRDF-based image formation model.  ...  Our challenge is to apply the deep neural network to the photometric stereo problem whose input is unstructured.  ... 
arXiv:1808.10093v1 fatcat:njutijy5jvfj3ml23kkqo7ksk4

Color photometric stereo and virtual image rendering using neural networks

Haruki Kawanaka, Yuji Iwahori, Robert J. Woodham, Kenji Funahashi
2007 Electronics & communications in Japan. Part 2, Electronics  
In this paper we extend the application of neural network-based photometric stereo founded on the principle of empirical photometric stereo to color images proposing a method for computing both the normal  ...  In addition, we propose a novel neural network-based rendering method that allows the generation of realistic virtual images of an object with arbitrary light source direction and from arbitrary viewpoints  ...  In previous work, self-calibration neural networkbased photometric stereo has been proposed as an attempt at finding a method that does not require a sphere [16] .  ... 
doi:10.1002/ecjb.20423 fatcat:wndo4r7shnaqziqcmmffln37pe

Multiview Textured Mesh Recovery by Differentiable Rendering [article]

Lixiang Lin, Jianke Zhu, Yisu Zhang
2022 arXiv   pre-print
To account for depth information, we optimize the shape geometry by minimizing the differences between the rendered mesh and the predicted depth from multi-view stereo.  ...  In contrast to the implicit neural representation on shape and color, we introduce a physically based inverse rendering scheme to jointly estimate the environment lighting and object's reflectance, which  ...  The pixel color of the rendered image Î is computed by the rendering equation Î(u,v) = L o (p, w o , T(u,v) ). ( 20 ) From the above all, the inverse rendering optimization can be directly self-supervised  ... 
arXiv:2205.12468v2 fatcat:y5kui3fbqzdn5owx7fsygugds4

PS-Transformer: Learning Sparse Photometric Stereo Network using Self-Attention Mechanism

Satoshi Ikehata
2021 British Machine Vision Conference  
To tackle this issue, this paper presents a deep sparse calibrated photometric stereo network named PS-Transformer which leverages the learnable self-attention mechanism to properly capture the complex  ...  Existing deep calibrated photometric stereo networks basically aggregate observations under different lights based on the pre-defined operations such as linear projection and max pooling.  ...  Since Woodham [27] proposed the first Lambertian calibrated photometric stereo algorithm, optimization based inverse rendering had been a mainstream approach [1, 9, 15, 16, 23] .  ... 
dblp:conf/bmvc/Ikehata21 fatcat:f2tdwdn7jrcpzhjumklcvrfeli

InverseRenderNet: Learning single image inverse rendering [article]

Ye Yu, William A. P. Smith
2018 arXiv   pre-print
We show how to train a fully convolutional neural network to perform inverse rendering from a single, uncontrolled image.  ...  By incorporating a differentiable renderer, our network can learn from self-supervision. Since the problem is ill-posed we introduce additional supervision: 1.  ...  From images with fixed viewpoint but varying illumination photometric stereo can be applied.  ... 
arXiv:1811.12328v1 fatcat:alsasb5xv5aelpsaxclhdsntrm
« Previous Showing results 1 — 15 out of 300 results