Adaptive Relaxed ADMM: Convergence Theory and Practical Implementation

Zheng Xu, Mario A. T. Figueiredo, Xiaoming Yuan, Christoph Studer, Tom Goldstein
2017 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
Many modern computer vision and machine learning ap- plications rely on solving difficult optimization problems that involve non-differentiable objective functions and constraints. The alternating direction method of multipliers (ADMM) is a widely used approach to solve such problems. Relaxed ADMM is a generalization of ADMM that often achieves better performance, but its efficiency depends strongly on algorithm parameters that must be chosen by an expert user. We propose an adaptive method
more » ... automatically tunes the key algorithm parameters to achieve optimal performance without user oversight. Inspired by recent work on adaptivity, the proposed adaptive relaxed ADMM (ARADMM) is derived by assuming a Barzilai-Borwein style linear gradient. A detailed convergence analysis of ARADMM is provided, and numerical results on several applications demonstrate fast practical convergence.
doi:10.1109/cvpr.2017.765 dblp:conf/cvpr/0002FYSG17 fatcat:dilvj65ayncxviltne44qxnikq