General Proximal Incremental Aggregated Gradient Algorithms: Better and Novel Results under General Scheme [article]

Tao Sun, Yuejiao Sun, Dongsheng Li, Qing Liao
<span title="2019-10-11">2019</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
The incremental aggregated gradient algorithm is popular in network optimization and machine learning research. However, the current convergence results require the objective function to be strongly convex. And the existing convergence rates are also limited to linear convergence. Due to the mathematical techniques, the stepsize in the algorithm is restricted by the strongly convex constant, which may make the stepsize be very small (the strongly convex constant may be small). In this paper, we
more &raquo; ... propose a general proximal incremental aggregated gradient algorithm, which contains various existing algorithms including the basic incremental aggregated gradient method. Better and new convergence results are proved even with the general scheme. The novel results presented in this paper, which have not appeared in previous literature, include: a general scheme, nonconvex analysis, the sublinear convergence rates of the function values, much larger stepsizes that guarantee the convergence, the convergence when noise exists, the line search strategy of the proximal incremental aggregated gradient algorithm and its convergence.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1910.05093v1">arXiv:1910.05093v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ag4xb5tjabg4xhav62rxzvmjzm">fatcat:ag4xb5tjabg4xhav62rxzvmjzm</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200930090403/https://arxiv.org/pdf/1910.05093v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/f0/a7/f0a71f31f9258483c013c0f47b00334da9b6f6e7.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1910.05093v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>