Filters








7,525 Hits in 3.8 sec

A General Analysis of the Convergence of ADMM [article]

Robert Nishihara, Laurent Lessard, Benjamin Recht, Andrew Packard, Michael I. Jordan
2015 arXiv   pre-print
We provide a new proof of the linear convergence of the alternating direction method of multipliers (ADMM) when one of the objective terms is strongly convex.  ...  On a numerical example, we demonstrate that minimizing the derived bound on the convergence rate provides a practical approach to selecting algorithm parameters for particular ADMM instances.  ...  FA9550-12-1-0339, NASA grant NRA NNX12AM55A, ONR grants N00014-11-1-0688 and N00014-14-1-0024, US ARL and US ARO grant W911NF-11-1-0391, NSF awards CCF-1359814 and CCF-1217058, NSF grant DGE-1106400, a  ... 
arXiv:1502.02009v3 fatcat:iv2lmcbpozaltlixrd4rw24rs4

On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks

Caihua Chen, Yuan Shen, Yanfei You
2013 Abstract and Applied Analysis  
For those problems, Han and Yuan (2012) have shown that the sequence generated by the alternating direction method of multipliers (ADMM) with three blocks converges globally to their KKT points under some  ...  Moreover, in order to accelerate the ADMM with three blocks, we also propose a relaxed ADMM involving an additional computation of optimal step size and establish its global convergence under mild conditions  ...  The second author is supported by university natural science research fund of jiangsu province under grant no. 13KJD110002. Abstract and Applied Analysis  ... 
doi:10.1155/2013/183961 fatcat:bbl4f32gcfbi5j2nx2flzceih4

A convergence proof of the split Bregman method for regularized least-squares problems [article]

Hung Nien, Jeffrey A. Fessler
2014 arXiv   pre-print
To have a concrete example, we conduct a convergence rate analysis of the ADMM algorithm using two splits for image restoration problems with quadratic data-fitting term and regularization term.  ...  In this paper, we show that when the data-fitting term is quadratic, the SB method is a convergent alternating direction method of multipliers (ADMM), and a straightforward convergence proof with inexact  ...  To verify our analysis, we conduct a convergence rate analysis of the ADMM algorithm with two split variables for image restoration problems with quadratic data-fitting term and regularization term.  ... 
arXiv:1402.4371v1 fatcat:dgapva6fpveyrpnyf2757lsym4

Splitting Methods for Convex Bi-Clustering and Co-Clustering [article]

Michael Weylandt
2019 arXiv   pre-print
in the primal update, and a three-block ADMM based on the operator splitting scheme of Davis and Yin.  ...  Co-Clustering, the problem of simultaneously identifying clusters across multiple aspects of a data set, is a natural generalization of clustering to higher-order structured data.  ...  COMPLEXITY ANALYSIS Both the ADMM and AMA are known to exhibit O(1/K) convergence in general, [13] , [25] , [26] , but the ADMM obtains a superior convergence rate of o(1/K) for strongly convex problems  ... 
arXiv:1901.06075v3 fatcat:x54bvj5znnedtk2pe77dmwcgtq

An Augmented ADMM Algorithm With Application to the Generalized Lasso Problem

Yunzhang Zhu
2017 Journal of Computational And Graphical Statistics  
We also consider a new varying penalty scheme for the ADMM algorithm, which could further accelerate the convergence, especially when solving a sequence of problems with tuning parameters of different  ...  Compared to a standard ADMM algorithm, our proposal significantly reduces the computational cost at each iteration while maintaining roughly the same overall convergence speed.  ...  We defer a detailed derivation and the analysis of its convergence properties until later sections.  ... 
doi:10.1080/10618600.2015.1114491 fatcat:7ecl65xdmfg5hira6n7aujx22e

Stochastic Variance-Reduced ADMM [article]

Shuai Zheng, James T. Kwok
2016 arXiv   pre-print
We also extend the proposed method for nonconvex problems, and obtain a convergence rate of O(1/T).  ...  The alternating direction method of multipliers (ADMM) is a powerful optimization solver in machine learning.  ...  [17] studied the convergence of the ADMM for solving certain nonconvex consensus and sharing problems, and showed that the generated sequence will converge to a stationary point, as well as a convergence  ... 
arXiv:1604.07070v3 fatcat:i7eqkowjybbr5pxkkvmtg6czxm

Dynamic Sharing Through the ADMM [article]

Xuanyu Cao, K.J. Ray Liu
2017 arXiv   pre-print
We analyze the convergence properties of the dynamic ADMM and show that, under several standard technical assumptions, the iterations of the dynamic ADMM converge linearly to some neighborhoods of the  ...  We also investigate the impact of the drifts on the steady state convergence behaviors of the dynamic ADMM.  ...  In Section III, theoretical analysis of the convergence properties of the dynamic ADMM is presented.  ... 
arXiv:1702.03874v2 fatcat:eax6co6qvbgz7pa3rcyqpe5j5i

Compositional Analysis of Hybrid Systems Defined Over Finite Alphabets [article]

Murat Cubuktepe, Mohamadreza Ahmadi, Ufuk Topcu, Brandon Hencey
2018 arXiv   pre-print
We consider the stability and the input-output analysis problems of a class of large-scale hybrid systems composed of continuous dynamics coupled with discrete dynamics defined over finite alphabets, e.g  ...  We show that the certificates of the method based on dissipativity theory can be computed by solving a set of semi-definite programs.  ...  NUMERICAL EXPERIMENTS In this section, we illustrate the proposed distributed analysis method with a large scale example, where we compare the convergence rate of ADMM with accelerated ADMM and several  ... 
arXiv:1803.00622v1 fatcat:ufyv3gbhjvg75k3v5n2u2w7faq

Differentially Private ADMM Algorithms for Machine Learning [article]

Tao Xu, Fanhua Shang, Yuanyuan Liu, Hongying Liu, Longjie Shen, Maoguo Gong
2020 arXiv   pre-print
Then we establish a new criterion to prove the convergence of the proposed algorithms including DP-ADMM. We also give the utility analysis of our DP-ADMM.  ...  From the viewpoint of theoretical analysis, we use the Gaussian mechanism and the conversion relationship between Rényi Differential Privacy (RDP) and DP to perform a comprehensive privacy analysis for  ...  ACKNOWLEDGMENTS We thank all the reviewers for their valuable comments. This work was supported by the National Natural Science  ... 
arXiv:2011.00164v1 fatcat:425slfilnze3nco2krv4nu4b24

Nonconvex Sparse Spectral Clustering by Alternating Direction Method of Multipliers and Its Convergence Analysis [article]

Canyi Lu, Jiashi Feng, Zhouchen Lin, Shuicheng Yan
2017 arXiv   pre-print
We propose an efficient Alternating Direction Method of Multipliers (ADMM) to solve the nonconvex SSC and provide the convergence guarantee.  ...  In particular, we prove that the sequences generated by ADMM always exist a limit point and any limit point is a stationary point.  ...  Main Result: The Convergence Analysis The most important contribution is the convergence analysis of the proposed ADMM in Algorithm 1 for nonconvex problems which is generally challenging.  ... 
arXiv:1712.02979v1 fatcat:sldsrqbkczbv5efxmebih64gj4

Convergence Analysis and Design of Multi-block ADMM via Switched Control Theory [article]

Jun Li, Hongfu Liu, Yue Wu, Yun Fu
2018 arXiv   pre-print
Second, we study exponential stability and stabilizability of the switched system for linear convergence analysis and design of ADMM by employing switched Lyapunov functions.  ...  Moreover, linear matrix inequalities conditions are proposed to ensure convergence of ADMM under arbitrary sequence, to find convergent sequences, and to design the fixed parameters.  ...  When N ≥ 3, however, the systematic convergence analysis of multi-block ADMM has been void for a long time.  ... 
arXiv:1709.05528v2 fatcat:wcqlmegtr5bgxdzgtgtkdrfns4

Fast Stochastic Alternating Direction Method of Multipliers [article]

Leon Wenliang Zhong, James T. Kwok
2013 arXiv   pre-print
This matches the convergence rate of the batch ADMM algorithm, but without the need to visit all the samples in each iteration.  ...  In this paper, we propose a new stochastic alternating direction method of multipliers (ADMM) algorithm, which incrementally approximates the full gradient in the linearized ADMM formulation.  ...  In this section, we show that it also has a much faster convergence rate. In the standard convergence analysis of ADMM, equation (4) is used for updating x [12] , [8] .  ... 
arXiv:1308.3558v1 fatcat:e7j43szn25b63gzdiwszg3wpya

A fundamental proof of convergence of alternating direction method of multipliers for weakly convex optimization

Tao Zhang, Zhengwei Shen
2019 Journal of Inequalities and Applications  
Specifically, we firstly show the convergence of the iterative sequences of the SWCCO-ADMM under a mild regularity condition; then we establish the o(1/k) sublinear convergence rate of the SWCCO-ADMM algorithm  ...  Due to the extensive applications of a weakly convex function in signal processing and machine learning, in this paper, we investigate the convergence of an ADMM algorithm to the strongly and weakly convex  ...  Sublinear and linear convergence rate analysis Compared with the large amount of convergence analysis results for convex/nonconvex ADMM algorithms, there are merely a limited number of references in the  ... 
doi:10.1186/s13660-019-2080-0 fatcat:bgm2msavabh7daz7coqaue2xge

Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge Computing [article]

Hao Chen, Yu Ye, Ming Xiao, Mikael Skoglund, H. Vincent Poor
2020 arXiv   pre-print
Given an appropriate mini-batch size, we show that the mini-batch stochastic ADMM based method converges in a rate of O(1/√(k)), where k denotes the number of iterations.  ...  A class of mini-batch stochastic alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.  ...  We note that Theorem 1 provides a sufficient condition to guarantee the convergence of the proposed sI-ADMM. csI-ADMM has the same convergence properties as those of sI-ADMM.  ... 
arXiv:2010.00914v1 fatcat:o7oy4w4hznehtok35l546kard4

On the Duality Gap Convergence of ADMM Methods [article]

Da Tang, Tong Zhang
2015 arXiv   pre-print
This paper provides a duality gap convergence analysis for the standard ADMM as well as a linearized version of ADMM.  ...  It is shown that under appropriate conditions, both methods achieve linear convergence. However, the standard ADMM achieves a faster accelerated convergence rate than that of the linearized ADMM.  ...  The authors would also like to thank Wotao Yin for helpful discussions.  ... 
arXiv:1508.03702v2 fatcat:sx3ohf5i35bztjsrmowaamy5ku
« Previous Showing results 1 — 15 out of 7,525 results