47 Hits in 2.8 sec

StyleGAN-induced data-driven regularization for inverse problems [article]

Arthur Conmy, Subhadip Mukherjee, Carola-Bibiane Schönlieb
2021 arXiv   pre-print
The ability of GANs to sample from high-dimensional distributions has naturally motivated researchers to leverage their power for modeling the image prior in inverse problems.  ...  Considering the inverse problems of image inpainting and super-resolution, we demonstrate that the proposed approach is competitive with, and sometimes superior to, state-of-the-art GAN-based image reconstruction  ...  Such data-driven regularizers have been shown to significantly outperform their hand-crafted variants on a wide array of imaging inverse problems.  ... 
arXiv:2110.03814v1 fatcat:gqatxjr6dnba3hkole6w5ycvwq

Comparing the latent space of generative models [article]

Andrea Asperti, Valerio Tonelli
2022 arXiv   pre-print
In this work we address the more general problem of comparing the latent spaces of different models, looking for transformations between them.  ...  We confined the investigation to the familiar and largely investigated case of generative models for the data manifold of human faces.  ...  Acknowledgements We would like to thank Fabio Merizzi for many interesting discussions on the subject of this article.  ... 
arXiv:2207.06812v1 fatcat:km7cqdqu3nad3ehcuul3gi4wbq

InvGAN: Invertible GANs [article]

Partha Ghosh, Dominik Zietlow, Michael J. Black, Larry S. Davis, Xiaochen Hu
2021 arXiv   pre-print
Recent progress in GANs have established them as an excellent choice for such tasks.  ...  StyleGAN) specific. These methods are nontrivial to extend to novel datasets or architectures. We propose a general framework that is agnostic to architecture and datasets.  ...  While PG and DZ are affiliated with Max Planck Institute for Intelligent Systems, this project was  ... 
arXiv:2112.04598v2 fatcat:ivzs54nkmzacxjwo66bmcrasuy

State-of-the-Art in the Architecture, Methods and Applications of StyleGAN [article]

Amit H. Bermano and Rinon Gal and Yuval Alaluf and Ron Mokady and Yotam Nitzan and Omer Tov and Or Patashnik and Daniel Cohen-Or
2022 arXiv   pre-print
Seeking to bring StyleGAN's latent control to real-world scenarios, the study of GAN inversion and latent space embedding has quickly gained in popularity.  ...  It aims to be of use for both newcomers, who wish to get a grasp of the field, and for more experienced readers that might benefit from seeing current research trends and existing tools laid out.  ...  Some propose latent code optimization and others apply data-driven inference.  ... 
arXiv:2202.14020v1 fatcat:qu3plbdnszdujcwxwq3zizysje

GAN Inversion: A Survey [article]

Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Hao Xue, Bolei Zhou, Ming-Hsuan Yang
2022 arXiv   pre-print
As an emerging technique to bridge the real and fake image domains, GAN inversion plays an essential role in enabling the pretrained GAN models such as StyleGAN and BigGAN to be used for real image editing  ...  GAN inversion aims to invert a given image back into the latent space of a pretrained GAN model, for the image to be faithfully reconstructed from the inverted code by the generator.  ...  Multimodal Learning For multimodal learning, several recent studies have focused on language-driven image generation and manipulation using StyleGAN. Xia et al.  ... 
arXiv:2101.05278v5 fatcat:ff3evb2nv5ezzaxju2cucbizde

Analyzing and Improving the Image Quality of StyleGAN [article]

Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila
2020 arXiv   pre-print
The style-based GAN architecture (StyleGAN) yields state-of-the-art results in data-driven unconditional generative image modeling.  ...  We furthermore visualize how well the generator utilizes its output resolution, and identify a capacity problem, motivating us to train larger models for additional quality improvements.  ...  Acknowledgements We thank Ming-Yu Liu for an early review, Timo Viitanen for help with the public release, David Luebke for in-depth discussions and helpful comments, and Tero Kuosmanen for technical support  ... 
arXiv:1912.04958v2 fatcat:sruxmi3uuvdhnanr6iehk3gnpe

Sound-Guided Semantic Image Manipulation [article]

Seung Hyun Lee, Wonseok Roh, Wonmin Byeon, Sang Ho Yoon, Chan Young Kim, Jinkyu Kim, Sangpil Kim
2021 arXiv   pre-print
We use a direct latent optimization method based on aligned embeddings for sound-guided image manipulation.  ...  For example, as shown in Fig. 3 , an audio sample a i forms a negative pair with âj for i = j, which induces a diffusive effect in the embedding space. Data Augmentation.  ...  In StyleGAN [20] , it is necessary to adaptively regularize style layer since each layer of latent code has different style attributes.  ... 
arXiv:2112.00007v1 fatcat:rcv7bt5ppvfihc57kqfa4d2pau

DeepLandscape: Adversarial Modeling of Landscape Video [article]

Elizaveta Logacheva, Roman Suvorov, Oleg Khomenko, Anton Mashikhin, Victor Lempitsky
2020 arXiv   pre-print
We propose simple but necessary modifications to StyleGAN inversion procedure, which lead to in-domain latent codes and allow to manipulate real images.  ...  Our architecture extends StyleGAN model by augmenting it with parts that allow to model dynamic changes in a scene.  ...  Data-driven estimation of homographies is out scope of this work, so we have prepared 12 homographies, one for each clock position (e.g. the "12h" move clouds up and towards the observer, the "3h" moves  ... 
arXiv:2008.09655v1 fatcat:un6qlzjwvve4jmzg5smoy6x52u

Multimodal Image Synthesis and Editing: A Survey [article]

Fangneng Zhan, Yingchen Yu, Rongliang Wu, Jiahui Zhang, Shijian Lu, Lingjie Liu, Adam Kortylewski, Christian Theobalt, Eric Xing
2022 arXiv   pre-print
As information exists in various modalities in real world, effective interaction and fusion among multimodal information plays a key role for the creation and perception of multimodal data in computer  ...  Instead of providing explicit guidance for network training, multimodal guidance offers intuitive and flexible means for image synthesis and editing.  ...  Specifically, TediGAN [27] maps paired images and texts into the common latent space of StyleGAN to achieve text-driven image generation.  ... 
arXiv:2112.13592v3 fatcat:46twjhz3hbe6rpm33k6ilnisga

SWAGAN: A Style-based Wavelet-driven Generative Model [article]

Rinon Gal, Dana Cohen, Amit Bermano, Daniel Cohen-Or
2021 arXiv   pre-print
Furthermore, we verify that our model's latent space retains the qualities that allow StyleGAN to serve as a basis for a multitude of editing tasks, and show that our frequency-aware approach also induces  ...  Even so, these networks still suffer from degradation in quality for high-frequency content, stemming from a spectrally biased architecture, and similarly unfavorable loss functions.  ...  Computational Performance and Visual Quality We first demonstrate our ability to capture highfrequency data on a set of toy problems.  ... 
arXiv:2102.06108v1 fatcat:zreadqhmufhsznwbkcb6gp2yrm

A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications [article]

Jie Gui, Zhenan Sun, Yonggang Wen, Dacheng Tao, Jieping Ye
2020 arXiv   pre-print
Finally, the future open research problems for GANs are pointed out.  ...  Thirdly, typical applications of GANs in image processing and computer vision, natural language processing, music, speech and audio, medical field, and data science are illustrated.  ...  The original GANs [3] can not learn the inverse mapping -projecting data back into the latent space. To solve this problem, Donahue et al.  ... 
arXiv:2001.06937v1 fatcat:4iqb2vnhezgjnphfv3taej7vbu


Feng-Lin Liu, Shu-Yu Chen, Yu-Kun Lai, Chunpeng Li, Yue-Ren Jiang, Hongbo Fu, Lin Gao
2022 ACM Transactions on Graphics  
ACKNOWLEDGMENTS We thank the anonymous reviewers for the constructive comments. This work was supported by grants from the National Natural Science Foundation of China (No. 62102403, No. 61872440  ...  The disentangled nature of W (or its extension W+) further induces awesome discoveries that real images could be projected back into the latent space for editing.  ...  , and solves an image-to-image translation problem.  ... 
doi:10.1145/3528223.3530056 fatcat:5hdq4e5efrhfzdcpljtsf2qsoq

Detection of AI-Generated Synthetic Faces [chapter]

Diego Gragnaniello, Francesco Marra, Luisa Verdoliva
2022 Advances in Computer Vision and Pattern Recognition  
Thanks to deep learning methods it is now possible to generate visual data with a high level of realism. This is especially true for human faces.  ...  In this chapter we will present the most effective techniques proposed in the literature for the detection of synthetic faces.  ...  They all use a neural classifier, eventually, but differ for the nature of the features on which the classification is based, handcrafted, or data-driven.  ... 
doi:10.1007/978-3-030-87664-7_9 fatcat:xgfieyw72jc4rivptsgekhrei4

High Resolution Face Editing with Masked GAN Latent Code Optimization [article]

Martin Pernuš, Vitomir Štruc, Simon Dobrišek
2021 arXiv   pre-print
The constraints are enforced with the help of an (differentiable) attribute classifier and face parser that provide the necessary reference information for the optimization procedure.  ...  that the proposed approach is able to edit face images with respect to several facial attributes with unprecedented image quality and at high-resolutions (1024x1024), while exhibiting considerably less problems  ...  data.  ... 
arXiv:2103.11135v2 fatcat:3gw3obsdonbjlgyqsujbbjqnie

Generative Adversarial Networks for Image Augmentation in Agriculture: A Systematic Review [article]

Ebenezer Olaniyi, Dong Chen, Yuzhen Lu, Yanbo Huang
2022 arXiv   pre-print
for data preparation, by algorithmically expanding training datasets.  ...  Beyond traditional data augmentation techniques, generative adversarial network (GAN) invented in 2014 in the computer vision community, provides a suite of novel approaches that can learn good data representations  ...  Training Instability Training GANs is prone significantly to instability problems Goodfellow et al., 2020) , despite a set of regularization techniques proposed for improving training stability (e.g.,  ... 
arXiv:2204.04707v2 fatcat:wcvmq3vl35fo7on2pqyblbzcku
« Previous Showing results 1 — 15 out of 47 results