1,817 Hits in 5.4 sec

De-rendering the World's Revolutionary Artefacts [article]

Shangzhe Wu and Ameesh Makadia and Jiajun Wu and Noah Snavely and Richard Tucker and Angjoo Kanazawa
2021 arXiv   pre-print
We conduct experiments on a real vase dataset and demonstrate compelling decomposition results, allowing for applications including free-viewpoint rendering and relighting.  ...  Recent works have shown exciting results in unsupervised image de-rendering -- learning to decompose 3D shape, appearance, and lighting from single-image collections without explicit supervision.  ...  Acknowledgements We would like to thank Christian Rupprecht, Soumyadip Sengupta, Manmohan Chandraker and Andrea Vedaldi for insightful discussions.  ... 
arXiv:2104.03954v2 fatcat:2yrcvmpemzglhphe6mhfidqvs4

Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model Using Deep Non-Rigid Structure From Motion

Mihir Sahasrabudhe, Zhixin Shu, Edward Bartrum, Riza Alp Guler, Dimitris Samaras, Iasonas Kokkinos
2019 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)  
We further use weak supervision to disentangle the non-rigid shape variability of human faces into identity and expression.  ...  We combine the 3D representation with a differentiable renderer to generate RGB * indicates equal contribution. images and append an adversarially trained refinement network to obtain sharp, photorealistic  ...  The scene factors that govern image formation primarily include surface geometry, camera position, material properties, and illumination.  ... 
doi:10.1109/iccvw.2019.00500 dblp:conf/iccvw/SahasrabudheSBG19 fatcat:f3dv3unkqnhvphph76laefew54

Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model using Deep Non-Rigid Structure from Motion [article]

Mihir Sahasrabudhe, Zhixin Shu, Edward Bartrum, Riza Alp Guler, Dimitris Samaras, Iasonas Kokkinos
2019 arXiv   pre-print
We further use weak supervision to disentangle the non-rigid shape variability of human faces into identity and expression.  ...  We exploit the 3D geometric nature of our model and use normal information to disentangle appearance into illumination, shading and albedo.  ...  The scene factors that govern image formation primarily include surface geometry, camera position, material properties and illumination.  ... 
arXiv:1904.11960v1 fatcat:ohpo2bu3rbe2hm5dwq4k55cnv4

Appearance Consensus Driven Self-Supervised Human Mesh Recovery [article]

Jogendra Nath Kundu, Mugalodi Rakesh, Varun Jampani, Rahul Mysore Venkatesh, R. Venkatesh Babu
2020 arXiv   pre-print
We present a self-supervised human mesh recovery framework to infer human pose and shape from monocular images in the absence of any paired supervision.  ...  The proposed FG appearance consistency objective makes use of a novel, differentiable Color-recovery module to obtain vertex colors without the need for any appearance network; via efficient realization  ...  color intensities for the rendering pipeline.  ... 
arXiv:2008.01341v1 fatcat:jyqhw3pwjrcxxpozwetd36r6ay

TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting [article]

Zhuoqian Yang, Wentao Zhu, Wayne Wu, Chen Qian, Qiang Zhou, Bolei Zhou, Chen Change Loy
2020 arXiv   pre-print
Without using any paired data for supervision, the proposed method can be trained in an unsupervised manner by exploiting invariance properties of three orthogonal factors of variation including motion  ...  Specifically, with loss functions carefully derived based on invariance, we train an auto-encoder to disentangle the latent representations of such factors given the source and target video clips.  ...  We would like to thank Tinghui Zhou, Rundi Wu and Kwan-Yee Lin for insightful discussion and their exceptional support.  ... 
arXiv:2003.14401v2 fatcat:ob5zqvucuvem5fvrmcfcczo2ei

Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer [article]

Wenzheng Chen, Jun Gao, Huan Ling, Edward J. Smith, Jaakko Lehtinen, Alec Jacobson, Sanja Fidler
2019 arXiv   pre-print
In this paper, we present {\emph DIB-R}, a differentiable rendering framework which allows gradients to be analytically computed for all pixels in an image.  ...  Enabling ML models to understand image formation might be key for generalization.  ...  In the fragment shader, pixel colors are decided by a combination of local properties including assigned vertex colors, textures, material properties, and lighting.  ... 
arXiv:1908.01210v2 fatcat:ewvmazpp65ccpmyc774y6exvvi

Unsupervised High-Fidelity Facial Texture Generation and Reconstruction [article]

Ron Slossberg, Ibrahim Jubran, Ron Kimmel
2021 arXiv   pre-print
In this paper, we propose a novel unified pipeline for both tasks, generation of both geometry and texture, and recovery of high-fidelity texture.  ...  Many methods have been proposed over the years to tackle the task of facial 3D geometry and texture recovery from a single image.  ...  Note that we utilize [7] for geometry recovery and thus focus our comparison on texture recovery only.  ... 
arXiv:2110.04760v1 fatcat:frsdjpfi65ei7atnfdyjogisn4

CPS++: Improving Class-level 6D Pose and Shape Estimation From Monocular Images With Self-Supervised Learning [article]

Fabian Manhardt and Gu Wang and Benjamin Busam and Manuel Nickel and Sven Meier and Luca Minciullo and Xiangyang Ji and Nassir Navab
2020 arXiv   pre-print
In essence, after training our proposed method fully supervised with synthetic data, we leverage recent advances in differentiable rendering to self-supervise the model with unannotated real RGB-D data  ...  To overcome this shortcoming, we additionally propose the idea of synthetic-to-real domain transfer for class-level 6D poses by means of self-supervised learning, which removes the burden of collecting  ...  We report individual results for each object in the supplementary material.  ... 
arXiv:2003.05848v3 fatcat:bvkodwdpnbe5jlcranexwwwv6i

Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding [article]

Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, Joshua B. Tenenbaum
2019 arXiv   pre-print
We marry two powerful ideas: deep representation learning for visual recognition and language understanding, and symbolic program execution for reasoning.  ...  data- and memory-efficient: it performs well after learning on a small number of training data; it can also encode an image into a compact representation, requiring less storage than existing methods for  ...  Acknowledgments We thank Jiayuan Mao, Karthik Narasimhan, and Jon Gauthier for helpful discussions and suggestions. We also thank Drew A. Hudson for sharing experimental results for comparison.  ... 
arXiv:1810.02338v2 fatcat:7zbzqjpvszg7lg7ftd5jfxix3e

Single-Shot Analysis of Refractive Shape Using Convolutional Neural Networks

Jonathan Stets, Zhengqin Li, Jeppe Revall Frisvad, Manmohan Chandraker
2019 2019 IEEE Winter Conference on Applications of Computer Vision (WACV)  
To accurately capture the image formation process, we use a physically-based renderer.  ...  In experiments, we extensively study the properties of our dataset and compare to baselines demonstrating its utility.  ...  Some of the models used for our dataset are from  ... 
doi:10.1109/wacv.2019.00111 dblp:conf/wacv/StetsLFC19 fatcat:tseciyahercf3gempbmzdqozha

MarrNet: 3D Shape Reconstruction via 2.5D Sketches [article]

Jiajun Wu, Yifan Wang, Tianfan Xue, Xingyuan Sun, William T Freeman, Joshua B Tenenbaum
2017 arXiv   pre-print
Our disentangled, two-step formulation has three advantages.  ...  Second, for 3D reconstruction from 2.5D sketches, systems can learn purely from synthetic data.  ...  Acknowledgements We thank Shubham Tulsiani for sharing the DRC results, and Chengkai Zhang for the help on shape visualization.  ... 
arXiv:1711.03129v1 fatcat:7nex5ycjmjhlvagd257okg64zq

Outdoor inverse rendering from a single image using multiview self-supervision [article]

Ye Yu, William A. P. Smith
2021 arXiv   pre-print
MVS depth also provides direct coarse supervision for normal map estimation. We believe this is the first attempt to use MVS supervision for learning inverse rendering.  ...  By incorporating a differentiable renderer, our network can learn from self-supervision. Since the problem is ill-posed we introduce additional supervision.  ...  The Titan Xp used for this research was donated by the NVIDIA Corporation.  ... 
arXiv:2102.06591v1 fatcat:ithclx2vzbaojpwihife3f2d2a

Learning Inverse Rendering of Faces from Real-world Videos [article]

Yuda Qiu, Zhangyang Xiong, Kai Han, Zhongyuan Wang, Zixiang Xiong, Xiaoguang Han
2020 arXiv   pre-print
Meanwhile, since no ground truth for any component is available for real images, it is not feasible to conduct supervised learning on real face images.  ...  In this paper we examine the problem of inverse rendering of real face images.  ...  Inverse rendering has important applications in image analysis (e.g., scene segmentation and material recognition) and editing (e.g., photo relighting).  ... 
arXiv:2003.12047v1 fatcat:g2oeps3csvf4niqucfxjxwrpwu

Three-dimensional Generative Adversarial Nets for Unsupervised Metal Artifact Reduction [article]

Megumi Nakao, Keiho Imanishi, Nobuhiro Ueda, Yuichiro Imai, Tadaaki Kirita, Tetsuya Matsuda
2020 arXiv   pre-print
We construct three-dimensional adversarial nets with a regularized loss function designed for metal artifacts from multiple dental fillings.  ...  Although there have been some studies on supervised metal artifact reduction through the learning of synthesized artifacts, it is difficult for simulated artifacts to cover the complexity of the real physical  ...  [36] proposed a CycleGAN-based artifact disentanglement network and compared quantitative evaluation results against existing supervised/unsupervised MAR methods using synthesized datasets.  ... 
arXiv:1911.08105v2 fatcat:zipfe5pnrze7novaaxwmlod4ma

Enforcing and Discovering Structure in Machine Learning [article]

Francesco Locatello
2021 arXiv   pre-print
It may be prudent to enforce corresponding structural properties to a learning algorithm's solution, such as incorporating prior beliefs, natural constraints, or causal structures.  ...  We observe that while the different methods successfully enforce properties "encouraged" by the corresponding losses, well-disentangled models seemingly cannot be identified without supervision.  ...  A prediction (object properties and position) is considered correct if there is a matching object with exactly the same properties (shape, material, color, and size) within a certain distance threshold  ... 
arXiv:2111.13693v1 fatcat:2urmfjeh6nhvjeiv3qoiy5hrum
« Previous Showing results 1 — 15 out of 1,817 results