A Neural Tangent Kernel Perspective of GANs [article]

Jean-Yves Franceschi
<span title="2022-02-02">2022</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
We propose a novel theoretical framework of analysis for Generative Adversarial Networks (GANs). We start by pointing out a fundamental flaw in previous theoretical analyses that leads to ill-defined gradients for the discriminator. We overcome this issue which impedes a principled study of GAN training, solving it within our framework by taking into account the discriminator's architecture. To this end, we leverage the theory of infinite-width neural networks for the discriminator via its
more &raquo; ... l Tangent Kernel. We characterize the trained discriminator for a wide range of losses and establish general differentiability properties of the network. Moreover, we derive new insights about the generated distribution's flow during training, advancing our understanding of GAN dynamics. We empirically corroborate these results via a publicly released analysis toolkit based on our framework, unveiling intuitions that are consistent with current GAN practice.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2106.05566v3">arXiv:2106.05566v3</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/7zlste4fcbfr5orbfuncqdw7ti">fatcat:7zlste4fcbfr5orbfuncqdw7ti</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20220212030034/https://arxiv.org/pdf/2106.05566v3.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/d8/a4/d8a47d4aaaff68f420656efc0bcaded3c22ccbca.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2106.05566v3" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>