66 Hits in 4.9 sec

On Quadratic Convergence of DC Proximal Newton Algorithm for Nonconvex Sparse Learning in High Dimensions [article]

Xingguo Li, Lin F. Yang, Jason Ge, Jarvis Haupt, Tong Zhang, Tuo Zhao
2018 arXiv   pre-print
We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions.  ...  Our proposed algorithm integrates the proximal Newton algorithm with multi-stage convex relaxation based on the difference of convex (DC) programming, and enjoys both strong computational and statistical  ...  Our Contribution: We study a multistage convex relaxation based proximal Newton algorithm for nonconvex regularized sparse learning.  ... 
arXiv:1706.06066v3 fatcat:vecf5urvlrgafp5nquvtbzxpuq

Composite Difference-Max Programs for Modern Statistical Estimation Problems [article]

Ying Cui, Jong-Shi Pang, Bodhisattva Sen
2018 arXiv   pre-print
We describe a nonmonotone majorization-minimization (MM) algorithm for solving the unified nonconvex, nondifferentiable optimization problem which is formulated as a specially structured composite dc program  ...  In our setup we allow all three component functions discussed above to be of the difference-of-convex (dc) type and illustrate them with a host of commonly used examples, including those in continuous  ...  Now we present the main result of this section on the subsequential convergence of {θ ν } to a d-stationary point of (8) , which generalizes the result in [47, Proposition 6] for the dc algorithm to  ... 
arXiv:1803.00205v3 fatcat:clfejbvgpnag3bof2hgdzycepy

A Fast Algorithm for Sparse Controller Design [article]

Matt Wytock, J. Zico Kolter
2013 arXiv   pre-print
We consider the task of designing sparse control laws for large-scale systems by directly minimizing an infinite horizon quadratic cost with an ℓ_1 penalty on the feedback controller gains.  ...  Our focus is on an improved algorithm that allows us to scale to large systems (i.e. those where sparsity is most useful) with convergence times that are several orders of magnitude faster than existing  ...  A FAST NEWTON-LASSO ALGORITHM A. Overview of the algorithm Our algorithm follows the overall structure of a "Newton-Lasso" method [23] , also sometimes called proximal Newton methods [24] .  ... 
arXiv:1312.4892v1 fatcat:cj6bauq6yzac7gpvtnxriqxdzq

Optimization with Sparsity-Inducing Penalties

Francis Bach
2011 Foundations and Trends® in Machine Learning  
However, for the sake of simplicity, we will keep this notation unchanged in the rest of the monograph.  ...  Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models.  ...  Acknowledgments All the authors would like to thank the anonymous reviewers, whose comments have greatly contributed to improve the quality of this monograph.  ... 
doi:10.1561/2200000015 fatcat:aycq5sjexfbk5olw2r5ausk4ae

A Unified Primal Dual Active Set Algorithm for Nonconvex Sparse Recovery [article]

Jian Huang, Yuling Jiao, Bangti Jin, Jin Liu, Xiliang Lu, Can Yang
2019 arXiv   pre-print
In this paper, we consider the problem of recovering a sparse signal based on penalized least squares formulations.  ...  We develop a novel algorithm of primal-dual active set type for a class of nonconvex sparsity-promoting penalties, including ℓ^0, bridge, smoothly clipped absolute deviation, capped ℓ^1 and minimax concavity  ...  Acknowledgements The authors are very grateful to the anonymous referee, the associate editor and the editor for their helpful comments, which have led to a significant improvement on the quality of the  ... 
arXiv:1310.1147v5 fatcat:keskxmncrrczbe2ksneb2kdl6m

Optimization Methods for Inverse Problems [article]

Nan Ye and Farbod Roosta-Khorasani and Tiangang Cui
2017 arXiv   pre-print
We then survey recent related advances in addressing similar challenges in problems faced by the machine learning community, and discuss their potential advantages for solving inverse problems.  ...  However, other, seemingly disjoint communities, such as that of machine learning, have developed, almost in parallel, interesting alternative methods which might have stayed under the radar of the inverse  ...  It requires solving a linear minimization problem over the feasible set, instead of a quadratic program as in the case of proximal gradient algorithms or projected gradient descent.  ... 
arXiv:1712.00154v1 fatcat:u4rhjnzzw5etje3f55jw6c4kom

A Semi-Smooth Newton Algorithm for High-Dimensional Nonconvex Sparse Learning [article]

Yueyong Shi, Jian Huang, Yuling Jiao, Qinglong Yang
2019 arXiv   pre-print
Simulation studies and a real data example suggest that our SSN algorithm, with comparable solution accuracy with the coordinate descent (CD) and the difference of convex (DC) proximal Newton algorithms  ...  In this paper we develop a fast algorithm for SCAD and MCP penalized learning problems. First, we show that the global minimizers of both models are roots of the nonsmooth equations.  ...  The authors also would like to thank Professor Defeng Sun for the helpful discussion on the algorithm.  ... 
arXiv:1802.08895v4 fatcat:dk72mwjrrfasbntvweh7ibensu

Optimization with Sparsity-Inducing Penalties [article]

Francis Bach, Rodolphe Jenatton, Julien Mairal, Guillaume Obozinski
2011 arXiv   pre-print
of experiments to compare various algorithms from a computational point of view.  ...  Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models.  ...  Acknowledgements All the authors would like to thank the anonymous reviewers, whose comments have greatly contributed to improve the quality of this paper.  ... 
arXiv:1108.0775v2 fatcat:ojhawgf3pnadfdiqk747gi25ry

A proximal MM method for the zero-norm regularized PLQ composite optimization problem [article]

Dongdong Zhang, Shaohua Pan, Shujun Bi
2020 arXiv   pre-print
Then, we propose a proximal majorization-minimization (MM) method, a convex relaxation approach not in the DC algorithm framework, for solving one of the DC surrogates which is a semiconvex PLQ minimization  ...  comparisons with a convergent indefinite-proximal ADMM for the partially smoothed DC surrogate verify its superiority in the quality of solutions and computing time.  ...  Convergence analysis of Algorithm 1 We shall follow the recipe of the convergence analysis in [2] for nonconvex nonsmooth optimization problems to establish the global and local linear convergence of  ... 
arXiv:2001.06176v1 fatcat:qdgfrbwv2zd4tc6ohwc2smav5i

On Iteratively Reweighted Algorithms for Nonsmooth Nonconvex Optimization in Computer Vision

Peter Ochs, Alexey Dosovitskiy, Thomas Brox, Thomas Pock
2015 SIAM Journal of Imaging Sciences  
Natural image statistics indicate that we should use nonconvex norms for most regularization tasks in image processing and computer vision.  ...  The efficiency and practical importance of the algorithm are demonstrated in computer vision tasks such as image denoising and optical flow.  ...  Acknowledgments The authors thank the anonymous reviewers for their valuable and constructive comments.  ... 
doi:10.1137/140971518 fatcat:o7w6em4dqjct5punkbtsgpr7wq

A Global Two-stage Algorithm for Non-convex Penalized High-dimensional Linear Regression Problems [article]

Peili Li, Min Liu, Zhou Yu
2021 arXiv   pre-print
In this paper, based on the DC (difference of convex functions) property of MCP and SCAD penalties, we aim to design a global two-stage algorithm for the high-dimensional least squares linear regression  ...  ) algorithm for solving non-convex penalized high-dimensional linear regression problems.  ...  Acknowledgements The work of Zhou Yu is supported in part by the National Natural Science Foundation of China (Grant No. 11971170).  ... 
arXiv:2111.11801v1 fatcat:fv26rgw4bffytk6xexqpjfgyhy

Proximal Algorithms in Statistics and Machine Learning [article]

Nicholas G. Polson, James G. Scott, Brandon T. Willard
2015 arXiv   pre-print
In this paper we develop proximal methods for statistical learning.  ...  Proximal point algorithms are useful in statistics and machine learning for obtaining optimization solutions for composite functions.  ...  We find that this combination of tools can be used together easily, applies to a broad range of functions and underlies many state-of-the-art approaches that scale well in high dimension.  ... 
arXiv:1502.03175v3 fatcat:264vfrtg3rgblpw2ak6vx3ztue

Cardinality Minimization, Constraints, and Regularization: A Survey [article]

Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, Alexandra Schwartz
2021 arXiv   pre-print
, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings.  ...  learning.  ...  High-Dimensional Statistics and Machine Learning.  ... 
arXiv:2106.09606v1 fatcat:jmsvxuqycvfz3gha4jv4f5seze

A Support Detection and Root Finding Approach for Learning High-dimensional Generalized Linear Models [article]

Jian Huang, Yuling Jiao, Lican Kang, Jin Liu, Yanyan Liu, Xiliang Lu
2020 arXiv   pre-print
In this paper, we develop a support detection and root finding procedure to learn the high dimensional sparse generalized linear models and denote this method by GSDAR.  ...  Feature selection is important for modeling high-dimensional data, where the number of variables can be much larger than the sample size.  ...  Acknowledgements The authors are grateful to the anonymous referees, the associate editor and the editor for their helpful comments, which have led to a significant improvement on the quality of the paper  ... 
arXiv:2001.05819v1 fatcat:ozsfhptlfzfwxfdyoyzyh3f4ry

Generalized Kalman Smoothing: Modeling and Algorithms [article]

A.Y. Aravkin, J.V. Burke, L. Ljung, A. Lozano, G. Pillonetto
2016 arXiv   pre-print
In contrast to classical models, these general estimators require use of iterated algorithms, and these have received increased attention from control, signal processing, machine learning, and optimization  ...  We discuss general statistical models for dynamic systems, making full use of nonsmooth convex penalties and constraints, and providing links to important models in signal processing and machine learning  ...  to high dimensions.  ... 
arXiv:1609.06369v2 fatcat:t4v5ertwwnh4pir334t5dkin6q
« Previous Showing results 1 — 15 out of 66 results