Accelerated Proximal Alternating Gradient-Descent-Ascent for Nonconvex Minimax Machine Learning [article]

Ziyi Chen, Shaocong Ma, Yi Zhou
2022 arXiv   pre-print
Alternating gradient-descent-ascent (AltGDA) is an optimization algorithm that has been widely used for model training in various machine learning applications, which aims to solve a nonconvex minimax optimization problem. However, the existing studies show that it suffers from a high computation complexity in nonconvex minimax optimization. In this paper, we develop a single-loop and fast AltGDA-type algorithm that leverages proximal gradient updates and momentum acceleration to solve
more » ... ed nonconvex minimax optimization problems. By leveraging the momentum acceleration technique, we prove that the algorithm converges to a critical point in nonconvex minimax optimization and achieves a computation complexity in the order of 𝒪(κ^11/6ϵ^-2), where ϵ is the desired level of accuracy and κ is the problem's condition number. Such a computation complexity improves the state-of-the-art complexities of single-loop GDA and AltGDA algorithms (see the summary of comparison in ). We demonstrate the effectiveness of our algorithm via an experiment on adversarial deep learning.
arXiv:2112.11663v7 fatcat:bgkeeeofijadzajcum44pbadsa