247 Hits in 6.2 sec

Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering [article]

Yuxuan Zhang, Wenzheng Chen, Huan Ling, Jun Gao, Yinan Zhang, Antonio Torralba, Sanja Fidler
2021 arXiv   pre-print
Differentiable rendering has paved the way to training neural networks to perform "inverse graphics" tasks such as predicting 3D geometry from monocular photographs.  ...  Key to our approach is to exploit GANs as a multi-view data generator to train an inverse graphics network using an off-the-shelf differentiable renderer, and the trained inverse graphics network as a  ...  Furthermore, our model achieves disentanglement in GANs and turns them into interpretable 3D neural renderers.  ... 
arXiv:2010.09125v2 fatcat:bxhd2qnncrgwfdsabvk542wsxa

GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields [article]

Michael Niemeyer, Andreas Geiger
2021 arXiv   pre-print
Combining this scene representation with a neural rendering pipeline yields a fast and realistic image synthesis model.  ...  Our key hypothesis is that incorporating a compositional 3D scene representation into the generative model leads to more controllable image synthesis.  ...  Image gans Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin- meet differentiable rendering for inverse graphics and inter- Brualla, Tomas Simon, Jason M.  ... 
arXiv:2011.12100v2 fatcat:bziey2wodzgknfznl3frlbnoii

HeadNeRF: A Real-time NeRF-based Parametric Head Model [article]

Yang Hong, Bo Peng, Haiyao Xiao, Ligang Liu, Juyong Zhang
2022 arXiv   pre-print
It can render high fidelity head images in real-time on modern GPUs, and supports directly controlling the generated images' rendering pose and various semantic attributes.  ...  To address this issue, we adopt the strategy of integrating 2D neural rendering to the rendering process of NeRF and design novel loss terms.  ...  However, these methods pay more attention to generating images that meet the specified distribution and lack semantic and interpretable control over the image synthesis.  ... 
arXiv:2112.05637v3 fatcat:ikjhucsrpva7rnysuzkz3zg65i

Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model using Deep Non-Rigid Structure from Motion [article]

Mihir Sahasrabudhe, Zhixin Shu, Edward Bartrum, Riza Alp Guler, Dimitris Samaras, Iasonas Kokkinos
2019 arXiv   pre-print
We combine the 3D representation with a differentiable renderer to generate RGB images and append an adversarially trained refinement network to obtain sharp, photorealistic image reconstruction results  ...  The learned generative model can be controlled in terms of interpretable geometry and appearance factors, allowing us to perform photorealistic image manipulation of identity, expression, 3D pose, and  ...  For this we combine the 3D lifting network with a differentiable renderer [34] , and bring the synthesized texture image in correspondence with the image coordinates.  ... 
arXiv:1904.11960v1 fatcat:ohpo2bu3rbe2hm5dwq4k55cnv4

Deep Generative Models in Engineering Design: A Review [article]

Lyle Regenwetter, Amin Heyrani Nobari, Faez Ahmed
2022 arXiv   pre-print
Recently, DGMs such as feedforward Neural Networks (NNs), Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and certain Deep Reinforcement Learning (DRL) frameworks have shown promising  ...  Anticipating continued growth, we conduct a review of recent advances to benefit researchers interested in DGMs for design.  ...  Advancements in 3D shape synthesis in the computer graphics community have leaned heavily on GANs or Autoencoders, as well as other machine learning advancements like Recurrent Neural Networks (RNNs),  ... 
arXiv:2110.10863v4 fatcat:zc4mo4nwzjdlne5jbvaetyugxy

BézierSketch: A generative model for scalable vector sketches [article]

Ayan Das, Yongxin Yang, Timothy Hospedales, Tao Xiang, Yi-Zhe Song
2020 arXiv   pre-print
The study of neural generative models of human sketches is a fascinating contemporary modeling problem due to the links between sketch image generation and the human drawing process.  ...  To this end, we first introduce a novel inverse graphics approach to stroke embedding that trains an encoder to embed each stroke to its best fit B\'ezier curve.  ...  Inverse Graphics "Inverse Graphics" is line of work that aims to estimate 3D scene parameters from raster images without supervision.  ... 
arXiv:2007.02190v2 fatcat:fwewez7kxjcphjyeuotwe3mlga

Path Tracing Denoising based on SURE Adaptive Sampling and Neural Network

Qiwei Xing, Chunyi Chen
2020 IEEE Access  
And the quality of final images is higher. • Compared with those methods which used the neural network to predict the kernel size of the filter, we introduce a set of additional features extracted from  ...  An anisotropic filter is used to reconstruct the final images with the parameters predicted by neural networks.  ...  His research interests include computer graphics, 3D visualization, and virtual reality.  ... 
doi:10.1109/access.2020.2999891 fatcat:l3efvetnefbptez3ejknvucvgy

Multimodal Image Synthesis and Editing: A Survey [article]

Fangneng Zhan, Yingchen Yu, Rongliang Wu, Jiahui Zhang, Shijian Lu
2021 arXiv   pre-print
We then describe multimodal image synthesis and editing approaches extensively with detailed frameworks including Generative Adversarial Networks (GANs), GAN Inversion, Transformers, and other methods  ...  As information exists in various modalities in real world, effective interaction and fusion among multimodal information plays a key role for the creation and perception of multimodal data in computer  ...  Yuan and Y. Peng, “Bridge-gan: Interpretable representation [100] H. Kim and A.  ... 
arXiv:2112.13592v1 fatcat:hxkfyxbtbfgltju323os3xompe

ICP [chapter]

2014 Computer Vision  
Definition An image decomposition is the result of a mathematical transformation of an image into a new set of images I 376 Image Enhancement and Restoration 6.  ...  Definition The purpose of illumination estimation is to determine the direction, intensity, and/or color of the lighting in a scene.  ...  Given the 3D models and the lighting conditions, novel views can be rendered using conventional graphic techniques.  ... 
doi:10.1007/978-0-387-31439-6_100030 fatcat:kfm7gu7zenb6tlz5qu26dvf3q4

Deep Image Synthesis from Intuitive User Input: A Review and Perspectives [article]

Yuan Xue, Yuan-Chen Guo, Han Zhang, Tao Xu, Song-Hai Zhang, Xiaolei Huang
2021 arXiv   pre-print
In many applications of computer graphics, art and design, it is desirable for a user to provide intuitive non-image input, such as text, sketch, stroke, graph or layout, and have a computer system automatically  ...  networks (GANs), variational autoencoders (VAEs), and flow-based methods have enabled more powerful and versatile image generation tasks.  ...  The authors also present a rendering model which first instantiates a 3D model by retrieving object meshes, then utilizes a differentiable renderer to render the corresponding semantic image and the depth  ... 
arXiv:2107.04240v2 fatcat:ticrsi27nzhozmw7dp7wwja2ni

Deep image synthesis from intuitive user input: A review and perspectives

Yuan Xue, Yuan-Chen Guo, Han Zhang, Tao Xu, Song-Hai Zhang, Xiaolei Huang
2021 Computational Visual Media  
AbstractIn many applications of computer graphics, art, and design, it is desirable for a user to provide intuitive non-image input, such as text, sketch, stroke, graph, or layout, and have a computer  ...  networks (GANs), variational autoencoders (VAEs), and flow-based methods have enabled more powerful and versatile image generation approaches.  ...  Joint Laboratory for Internet Innovation Technology.  ... 
doi:10.1007/s41095-021-0234-8 fatcat:ot6dyrrrsnakxob4jzw4zld7zu

Design Target Achievement Index: A Differentiable Metric to Enhance Deep Generative Models in Multi-Objective Inverse Design [article]

Lyle Regenwetter, Faez Ahmed
2022 arXiv   pre-print
and specialized tabular generation algorithms like the Conditional Tabular GAN (CTGAN).  ...  While early works are promising, further advancement will depend on addressing several critical considerations such as design quality, feasibility, novelty, and targeted inverse design.  ...  We also acknowledge MathWorks for supporting this research.  ... 
arXiv:2205.03005v1 fatcat:ouy7t7weabd4vitkm5jkwtfsre

2021 Index IEEE Transactions on Visualization and Computer Graphics Vol. 27

2022 IEEE Transactions on Visualization and Computer Graphics  
Departments and other items may also be covered if they have been judged to have archival value. The Author Index contains the primary entry for each item, listed under the first author's name.  ...  The primary entry includes the coauthors' names, the title of the paper or other item, and its location, specified by the publication abbreviation, year, month, and inclusive pagination.  ...  ., +, TVCG Sept. 2021 3670-3684 Computer graphics Attribute-Conditioned Layout GAN for Automatic Graphic Design.  ... 
doi:10.1109/tvcg.2022.3163599 fatcat:2mtpsecojbc33pqht3n7oyqmoq

Smart Cameras [article]

David J. Brady, Minghao Hu, Chengyu Wang, Xuefei Yan, Lu Fang, Yiwnheng Zhu, Yang Tan, Ming Cheng, Zhan Ma
2020 arXiv   pre-print
Modern cameras use physical components and software to capture, compress and display image data.  ...  Deep learning enables 10-100x reduction in electrical sensor power per pixel, 10x improvement in depth of field and dynamic range and 10-100x improvement in image pixel count.  ...  This means, for example, that neural algorithms can generate color images from monochrome [296] , generate 3D scenes from 2D photographs [297] and generate images of horses from images of zebra [298  ... 
arXiv:2002.04705v1 fatcat:277yq2oaujdoxbtqqsq6naodma

Modern Augmented Reality: Applications, Trends, and Future Directions [article]

Shervin Minaee, Xiaodan Liang, Shuicheng Yan
2022 arXiv   pre-print
We then give an overview of around 100 recent promising machine learning based works developed for AR systems, such as deep learning works for AR shopping (clothing, makeup), AR based image filters (such  ...  Augmented reality (AR) is one of the relatively old, yet trending areas in the intersection of computer vision and computer graphics with numerous applications in several areas, from gaming and entertainment  ...  ACKNOWLEDGMENTS We would like to thank Iasonas Kokkinos, Qi Pan, Lyric Kaplan, and Liz Markman for reviewing this work, and providing very helpful comments and suggestions.  ... 
arXiv:2202.09450v2 fatcat:x436ycnvxnhdpfdvhnxkzgbqce
« Previous Showing results 1 — 15 out of 247 results