1,429 Hits in 5.8 sec

Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis [article]

Xihui Liu, Guojun Yin, Jing Shao, Xiaogang Wang, Hongsheng Li
2020 arXiv   pre-print
In order to better exploit the semantic layout for the image generator, we propose to predict convolutional kernels conditioned on the semantic label map to generate the intermediate feature maps from  ...  Semantic image synthesis aims at generating photorealistic images from semantic layouts.  ...  We thank Lu Sheng for proofreading and helpful suggestions on the paper.  ... 
arXiv:1910.06809v3 fatcat:cttcvpisnbeufe3dt6myawin3i

Image Synthesis via Semantic Composition [article]

Yi Wang, Lu Qi, Ying-Cong Chen, Xiangyu Zhang, Jiaya Jia
2021 arXiv   pre-print
In this paper, we present a novel approach to synthesize realistic images based on their semantic layouts. It hypothesizes that for objects with similar appearance, they share similar representation.  ...  Conditioning on these features, we propose a dynamic weighted network constructed by spatially conditional computation (with both convolution and normalization).  ...  This new representation is also beneficial to unpaired image-to-image translation. We will study its applicability to other generation tasks in the future.  ... 
arXiv:2109.07053v1 fatcat:tvs264q3djcsjnr4chmz3ralra

Global-Affine and Local-Specific Generative Adversarial Network for semantic-guided image generation

Susu Zhang, Jiancheng Ni, Lijun Hou, Zili Zhou, Jie Hou, Feng Gao
2021 Mathematical Foundations of Computing  
The recent progress in learning image feature representations has opened the way for tasks such as label-to-image or text-to-image synthesis.  ...  To achieve this, we adopt the graph convolutional network to calculate the instance locations and spatial relationships from scene graphs, which allows our model to obtain the highfidelity semantic layouts  ...  There are a variety of input types in conditional image synthesis. Tang et al.  ... 
doi:10.3934/mfc.2021009 fatcat:wvg4u5kxsnay7jyfycuw5vixye

Learning Hierarchical Semantic Image Manipulation through Structured Representations [article]

Seunghoon Hong and Xinchen Yan and Thomas Huang and Honglak Lee
2018 arXiv   pre-print
Key to our hierarchical framework is that we employ a structured semantic layout as our intermediate representation for manipulation.  ...  In this work, we present a novel hierarchical framework for semantic image manipulation.  ...  [11] employed a convolutional encoder-decoder network with conditional adversarial objective to learn label-to-pixel mapping.  ... 
arXiv:1808.07535v2 fatcat:yg3kunbzt5cbdhpa3pu3a5nmrq

Synthetic Convolutional Features for Improved Semantic Segmentation [article]

Yang He and Bernt Schiele and Mario Fritz
2020 arXiv   pre-print
Recently, learning-based image synthesis has enabled to generate high-resolution images, either applying popular adversarial training or a powerful perceptual loss.  ...  However, it remains challenging to successfully leverage synthetic data for improving semantic segmentation with additional synthetic images.  ...  We aim to learn a semantic segmentation model with a mixture of real images and synthetic data from a generator, allowing to sample paired data from semantic layout masks, which assign categories for each  ... 
arXiv:2009.08849v1 fatcat:dn5gxjhu7jaorep3sf37adtgpq

End-to-End Optimization of Scene Layout [article]

Andrew Luo, Zhoutong Zhang, Jiajun Wu, Joshua B. Tenenbaum
2020 arXiv   pre-print
We propose an end-to-end variational generative model for scene layout synthesis conditioned on scene graphs.  ...  Given a depth and a semantics map, the differentiable rendering module enables optimizing over the synthesized layout to fit the given input in an analysis-by-synthesis fashion.  ...  Graph Convolutional Network MLP Predicted Layout (a) Testing Input Scene Graph Learned Layout Latent Graph Convolutional Network MLP Predicted Layout Input Scene Graph  ... 
arXiv:2007.11744v1 fatcat:lli4vd4kn5bghot2vq6zyrf5le

Diverse Image Synthesis from Semantic Layouts via Conditional IMLE [article]

Ke Li, Tianhao Zhang, Jitendra Malik
2019 arXiv   pre-print
Most existing methods for conditional image synthesis are only able to generate a single plausible image for any given input, or at best a fixed number of plausible images.  ...  same semantic layout.  ...  ,m || y θ j − y i || 2 2 To apply IMLE to conditional image synthesis, we need to model all the different distributions p( I|L, θ) for different semantic layouts L.  ... 
arXiv:1811.12373v2 fatcat:7o6lhf6xr5d65n7ujgapcgl4pu

Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis [article]

Seunghoon Hong and Dingdong Yang and Jongwook Choi and Honglak Lee
2018 arXiv   pre-print
We propose a novel hierarchical approach for text-to-image synthesis by inferring semantic layout.  ...  Instead of learning a direct mapping from text to image, our algorithm decomposes the generation process into multiple steps, in which it first constructs a semantic layout from the text by the layout  ...  We investigate a way to improve text-to-image synthesis on general images, by conditioning generation on inferred semantic layout.  ... 
arXiv:1801.05091v2 fatcat:743uxg4geneqdds5ws6zmnigmu

Semantic Image Synthesis with Spatially-Adaptive Normalization [article]

Taesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu
2019 arXiv   pre-print
We propose spatially-adaptive normalization, a simple but effective layer for synthesizing photorealistic images given an input semantic layout.  ...  To address the issue, we propose using the input layout for modulating the activations in normalization layers through a spatially-adaptive, learned transformation.  ...  Taesung Park contributed to the work during his internship at NVIDIA. His Ph.D. is supported by a Samsung Scholarship.  ... 
arXiv:1903.07291v2 fatcat:ym2enfzlnbcp5e2znpvpjflvuy

Image Synthesis From Reconfigurable Layout and Style [article]

Wei Sun, Tianfu Wu
2019 arXiv   pre-print
Despite remarkable recent progress on both unconditional and conditional image synthesis, it remains a long-standing problem to learn generative models that are capable of synthesizing realistic and sharp  ...  In this paper, we present a layout- and style-based architecture for generative adversarial networks (termed LostGANs) that can be trained end-to-end to generate images from reconfigurable layout and style  ...  Acknowledgement The authors would like to thank the anonymous reviewers for their helpful comments.  ... 
arXiv:1908.07500v1 fatcat:hzvw33cy4vck5ahwnvmjg6e6ru

Deep Image Synthesis from Intuitive User Input: A Review and Perspectives [article]

Yuan Xue, Yuan-Chen Guo, Han Zhang, Tao Xu, Song-Hai Zhang, Xiaolei Huang
2021 arXiv   pre-print
In many applications of computer graphics, art and design, it is desirable for a user to provide intuitive non-image input, such as text, sketch, stroke, graph or layout, and have a computer system automatically  ...  This paper reviews recent works for image synthesis given intuitive user input, covering advances in input versatility, image generation methodology, benchmark datasets, and evaluation metrics.  ...  Thus they proposed Conditional Convolution Blocks (CC Block), where parameters for convolution kernels are predicted from semantic layouts.  ... 
arXiv:2107.04240v2 fatcat:ticrsi27nzhozmw7dp7wwja2ni

Boosting Image Outpainting with Semantic Layout Prediction [article]

Ye Ma, Jin Ma, Min Zhou, Quan Chen, Tiezheng Ge, Yuning Jiang, Tong Lin
2021 arXiv   pre-print
Secondly, another GAN model is trained to synthesize real images based on the extended semantic layouts.  ...  Firstly, we train a GAN to extend regions in semantic segmentation domain instead of image domain.  ...  semantic layout prediction.  ... 
arXiv:2110.09267v1 fatcat:22z45o6ciraplodvzwyi5w5q2i

Deep image synthesis from intuitive user input: A review and perspectives

Yuan Xue, Yuan-Chen Guo, Han Zhang, Tao Xu, Song-Hai Zhang, Xiaolei Huang
2021 Computational Visual Media  
AbstractIn many applications of computer graphics, art, and design, it is desirable for a user to provide intuitive non-image input, such as text, sketch, stroke, graph, or layout, and have a computer  ...  This paper reviews recent works for image synthesis given intuitive user input, covering advances in input versatility, image generation methodology, benchmark datasets, and evaluation metrics.  ...  National Natural Science Foundation of China (Project Nos. 61521002 and 61772298), a Research Grant of Beijing Higher Institution Engineering Research Center, and the Tsinghua-Tencent Joint Laboratory for  ... 
doi:10.1007/s41095-021-0234-8 fatcat:ot6dyrrrsnakxob4jzw4zld7zu

Realistic Image Synthesis with Configurable 3D Scene Layouts [article]

Jaebong Jeong, Janghun Jo, Jingdong Wang, Sunghyun Cho, Jaesik Park
2021 arXiv   pre-print
Recent conditional image synthesis approaches provide high-quality synthesized images.  ...  To provide users with rich controllability on synthesized images in the aspect of 3D geometry, we propose a novel approach to realistic-looking image synthesis based on a configurable 3D scene layout.  ...  Introduction Conditional image synthesis aims to synthesize realistic image conditioning on user input to control the generation process.  ... 
arXiv:2108.10031v2 fatcat:j435aam32jfxtg46j7jmtaxfwe

Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts [article]

Levent Karacan, Zeynep Akata, Aykut Erdem, Erkut Erdem
2016 arXiv   pre-print
In this work, we propose a novel deep conditional generative adversarial network architecture that takes its strength from the semantic layout and scene attributes integrated as conditioning variables.  ...  We show that our architecture is able to generate realistic outdoor scene images under different conditions, e.g. day-night, sunny-foggy, with clear object boundaries.  ...  and a convolutional discriminator network learns to determine if an image is real or fake.  ... 
arXiv:1612.00215v1 fatcat:cq4busmwhbdptetzkq75muad7u
« Previous Showing results 1 — 15 out of 1,429 results