22,960 Hits in 7.8 sec

Large-scale randomized-coordinate descent methods with non-separable linear constraints [article]

Sashank Reddi, Ahmed Hefny, Carlton Downey, Avinava Dubey, Suvrit Sra
2015 arXiv   pre-print
We develop randomized (block) coordinate descent (CD) methods for linearly constrained convex optimization.  ...  Unlike most CD methods, we do not assume the constraints to be separable, but let them be coupled linearly.  ...  DISCUSSION AND FUTURE WORK We presented randomized coordinate descent methods for solving convex optimization problems with linear constraints that couple the variables.  ... 
arXiv:1409.2617v5 fatcat:czfw4iwikzaapfl2tu5zw3hnpa

Randomized sketch descent methods for non-separable linearly constrained optimization [article]

Ion Necoara, Martin Takac
2018 arXiv   pre-print
In this paper we consider large-scale smooth optimization problems with multiple linear coupled constraints.  ...  From our knowledge, this is the first convergence analysis of random sketch descent algorithms for optimization problems with multiple non-separable linear constraints.  ...  Our approach introduces general sketch descent algorithms for solving large-scale smooth optimization problems with multiple linear coupled constraints.  ... 
arXiv:1808.02530v1 fatcat:pngicbwfojb7re7hi7kuj4hb6m

A Primer on Coordinate Descent Algorithms [article]

Hao-Jun Michael Shi, Shenyinying Tu, Yangyang Xu, Wotao Yin
2017 arXiv   pre-print
This particular class of algorithms has recently gained popularity due to their effectiveness in solving large-scale optimization problems in machine learning, compressed sensing, image processing, and  ...  Coordinate descent algorithms solve optimization problems by successively minimizing along each coordinate or coordinate hyperplane, which is ideal for parallelized and distributed computing.  ...  Introduction 1.Overview This monograph discusses a class of algorithms, called coordinate descent (CD) algorithms, which is useful in solving large-scale optimization problems with smooth or non-smooth  ... 
arXiv:1610.00040v2 fatcat:fo3xzcsx4rb4xauip34j5jbm3y

Stochastic Parallel Block Coordinate Descent for Large-scale Saddle Point Problems [article]

Zhanxing Zhu, Amos J. Storkey
2015 arXiv   pre-print
We propose an efficient stochastic block coordinate descent method using adaptive primal-dual updates, which enables flexible parallel optimization for large-scale problems.  ...  Our method shares the efficiency and flexibility of block coordinate descent methods with the simplicity of primal-dual methods and utilizing the structure of the separable convex-concave saddle point  ...  Inspired by the recent success of coordinate descent methods for solving separable optimization problems, we incorporate a stochastic block coordinate descent technique into above primal-dual methods and  ... 
arXiv:1511.07294v1 fatcat:tssvaw6u6nfihfvm4moear5kwi

A generic coordinate descent solver for nonsmooth convex optimization [article]

Olivier Fercoq
2019 arXiv   pre-print
We present a generic coordinate descent solver for the minimization of a nonsmooth convex objective with structure. The method can deal in particular with problems with linear constraints.  ...  So, the algorithm can be used to solve a large variety of problems including Lasso, sparse multinomial logistic regression, linear and quadratic programs.  ...  Paper Non-separable non-smooth functions Non-separable non-smooth objective functions are much more challenging to coordinate descent methods.  ... 
arXiv:1812.00628v2 fatcat:ebvpesd2bbfinog3glef53x62m

Convex Optimization for Big Data: Scalable, randomized, and parallel algorithms for big data analytics

Volkan Cevher, Stephen Becker, Mark Schmidt
2014 IEEE Signal Processing Magazine  
We provide an overview of this emerging field, describe contemporary approximation techniques like first-order methods and randomization for scalability, and survey the important role of parallel and distributed  ...  Big Data scaling via randomization In theory, first-order methods are well-positioned to address very large-scale problems.  ...  Stochastic gradient methods In contrast to randomized coordinate descent methods, which update a single coordinate at a time with its exact gradient, stochastic gradient methods update all coordinates  ... 
doi:10.1109/msp.2014.2329397 fatcat:7np3knuhena2fd5o6tqjtpbzai

Random Block Coordinate Descent Methods for Linearly Constrained Optimization over Networks

Ion Necoara, Yurii Nesterov, François Glineur
2017 Journal of Optimization Theory and Applications  
In this paper we develop random block coordinate descent methods for minimizing large-scale linearly constrained convex problems over networks.  ...  Contributions: In this paper we develop random (block) coordinate descent methods with fixed probability distributions for choosing the (block of) coordinates, suited for large-scale optimization problems  ...  In contrast, in this paper we consider coordinate descent methods for convex problems with linear coupled constraints.  ... 
doi:10.1007/s10957-016-1058-z fatcat:bgk377dzkvfxlefygsophxqlli

Faster Parallel Solver for Positive Linear Programs via Dynamically-Bucketed Selective Coordinate Descent [article]

Di Wang, Michael Mahoney, Nishanth Mohan, Satish Rao
2015 arXiv   pre-print
To achieve our improvement, we introduce an algorithmic technique of broader interest: dynamically-bucketed selective coordinate descent (DB-SCD).  ...  More generally, this method addresses "interference" among coordinates, by which we mean the impact of the update of one coordinate on the gradients of other coordinates.  ...  Then y − OPT is a non-negative random variable with expectation Remark.  ... 
arXiv:1511.06468v1 fatcat:nybphd27erbl7fuwiuepmr32wa

Coordinate Descent with Online Adaptation of Coordinate Frequencies [article]

Tobias Glasmachers, Ürün Dogan
2014 arXiv   pre-print
We consider general CD with non-uniform selection of coordinates.  ...  Coordinate descent (CD) algorithms have become the method of choice for solving a number of optimization problems in machine learning.  ...  Natural competitors for solving large scale convex problems are (trust region/pseudo) Newton methods and (stochastic) gradient descent.  ... 
arXiv:1401.3737v1 fatcat:xlcvbr7apfbphef5yaa2io5qhm

Bridging the Gap Between Adversarial Robustness and Optimization Bias [article]

Fartash Faghri, Sven Gowal, Cristina Vasconcelos, David J. Fleet, Fabian Pedregosa, Nicolas Le Roux
2021 arXiv   pre-print
To the best of our knowledge, this work is the first to study the impact of optimization methods such as sign gradient descent and proximal methods on adversarial robustness.  ...  Second, we characterize the robustness of linear convolutional models, showing that they resist attacks subject to a constraint on the Fourier-ℓ_∞ norm.  ...  Maximum margin classification does not require linear separability, because there can exist a classifier with ε < 0 that satisfies the margin constraints.  ... 
arXiv:2102.08868v2 fatcat:qhty2vogyvdmzaza45yr2xpq4y

A Block-Coordinate Descent Approach for Large-scale Sparse Inverse Covariance Estimation

Eran Treister, Javier Turek
2014 Neural Information Processing Systems  
In this paper we present a new block-coordinate descent approach for solving the problem for large-scale data sets.  ...  Numerical experiments on both synthetic and real gene expression data demonstrate that our approach outperforms the existing state of the art methods, especially for large-scale problems. * The authors  ...  For large-scale problems, we compare our method only with BIG-QUIC as it is the only feasible method known to us at this scale.  ... 
dblp:conf/nips/TreisterT14 fatcat:jzsgg5l6gfhbbndkutfnq4p3zi

Speeding-Up Convergence via Sequential Subspace Optimization: Current State and Future Directions [article]

Michael Zibulevsky
2013 arXiv   pre-print
We explored its combination with Parallel Coordinate Descent and Separable Surrogate Function methods, obtaining state of the art results in above-mentioned areas.  ...  One can also accelerate Augmented Lagrangian method for constrained optimization problems and Alternating Direction Method of Multipliers for problems with separable objective function and non-separable  ...  Another task is to accelerate Augmented Lagrangian method for constrained optimization problems and Alternating Direction Method of Multipliers for problems with separable objective function and non-separable  ... 
arXiv:1401.0159v1 fatcat:j4pysvjrqjfz7ozapzxpn7tj4a

BILGO: Bilateral greedy optimization for large scale semidefinite programming

Zhifeng Hao, Ganzhao Yuan, Bernard Ghanem
2014 Neurocomputing  
Extensive experimental results clearly demonstrate that BILGO can solve large-scale semidefinite  ...  Moreover, BILGO can be easily extended to handle low rank constraints.  ...  Connection with Random Conic Pursuit: Random Conic Pursuit was proposed in [18] for SDP problems with linear constraints.  ... 
doi:10.1016/j.neucom.2013.07.024 fatcat:vw6tiehxljanvgdkzlps7i4wuy

Let's Make Block Coordinate Descent Converge Faster: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence [article]

Julie Nutini and Issam Laradji and Mark Schmidt
2022 arXiv   pre-print
Block coordinate descent (BCD) methods are widely used for large-scale numerical optimization because of their cheap iteration costs, low memory requirements, amenability to parallelization, and ability  ...  for certain problems with sparse solutions (and in some cases finite termination at an optimal solution).  ...  Warren Hare for fruitful conversations that helped with Section 6.  ... 
arXiv:1712.08859v3 fatcat:a4j6jaf5ajh4birmb3st4biv5a

Tightening Fractional Covering Upper Bounds on the Partition Function for High-Order Region Graphs [article]

Tamir Hazan, Jian Peng, Amnon Shashua
2012 arXiv   pre-print
This is a key for computational relevancy for large problems with thousands of regions.  ...  To solve these programs effectively for general region graphs we utilize the entropy barrier method, thus decomposing the original programs by their dual programs and solve them with dual block optimization  ...  with large-scale and high-order graphical models.  ... 
arXiv:1210.4881v1 fatcat:k2j6r3lervhp7hi6goziopy6ou
« Previous Showing results 1 — 15 out of 22,960 results