A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit <a rel="external noopener" href="https://arxiv.org/pdf/1807.03907v1.pdf">the original URL</a>. The file type is <code>application/pdf</code>.
The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization
[article]
<span title="2018-07-11">2018</span>
<i >
arXiv
</i>
<span class="release-stage" >pre-print</span>
Preserved Fulltext
Motivated by applications in Optimization, Game Theory, and the training of Generative Adversarial Networks, the convergence properties of first order methods in min-max problems have received extensive study. It has been recognized that they may cycle, and there is no good understanding of their limit points when they do not. When they converge, do they converge to local min-max solutions? We characterize the limit points of two basic first order methods, namely Gradient Descent/Ascent (GDA)
<span class="external-identifiers">
<a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1807.03907v1">arXiv:1807.03907v1</a>
<a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/7mmpjo7z7za33j2ak6gtd3tpfy">fatcat:7mmpjo7z7za33j2ak6gtd3tpfy</a>
</span>
more »
... d Optimistic Gradient Descent Ascent (OGDA). We show that both dynamics avoid unstable critical points for almost all initializations. Moreover, for small step sizes and under mild assumptions, the set of {OGDA}-stable critical points is a superset of {GDA}-stable critical points, which is a superset of local min-max solutions (strict in some cases). The connecting thread is that the behavior of these dynamics can be studied from a dynamical systems perspective.
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200910081320/https://arxiv.org/pdf/1807.03907v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext">
<button class="ui simple right pointing dropdown compact black labeled icon button serp-button">
<i class="icon ia-icon"></i>
Web Archive
[PDF]
<div class="menu fulltext-thumbnail">
<img src="https://blobs.fatcat.wiki/thumbnail/pdf/75/b3/75b32165fd7d2aa2f9881005db72d5cea1b94d4a.180px.jpg" alt="fulltext thumbnail" loading="lazy">
</div>
</button>
</a>
<a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1807.03907v1" title="arxiv.org access">
<button class="ui compact blue labeled icon button serp-button">
<i class="file alternate outline icon"></i>
arxiv.org
</button>
</a>