104 Hits in 4.5 sec

Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization

Martin Jaggi
2013 International Conference on Machine Learning  
We provide stronger and more general primal-dual convergence results for Frank-Wolfe-type algorithms (a.k.a. conditional gradient) for constrained convex optimization, enabled by a simple framework of  ...  We present a new general framework for convex optimization over matrix factorizations, where every Frank-Wolfe iteration will consist of a low-rank update, and discuss the broad application areas of this  ...  MJ acknowledges support by the ERC Project SIPA, by a Google research award, and by the Swiss National Science Foundation (SNSF). Most of this work was done while at ETH Zurich.  ... 
dblp:conf/icml/Jaggi13 fatcat:sjrno5eynbf6jcn5o4ckcdf344

Revisiting the Approximate Carathéodory Problem via the Frank-Wolfe Algorithm [article]

Cyrille W. Combettes, Sebastian Pokutta
2021 arXiv   pre-print
Lastly, we address the problem of finding sparse approximate projections onto 𝒞 in the ℓ_p-norm, p∈[1,+∞].  ...  We revisit the approximate Carathéodory problem by solving the primal problem via the Frank-Wolfe algorithm, providing a simplified analysis and leading to an efficient practical method.  ...  Revisiting Frank-Wolfe: Projection-free sparse convex optimization. In Proceedings of the 30th International Conference on Machine Learning, pages 427–435, 2013. [23] T.  ... 
arXiv:1911.04415v5 fatcat:ymeeb5d4bjezre7cw43rn3vyuy

How Does Momentum Help Frank Wolfe? [article]

Bingcong Li, Mario Coutino, Georgios B. Giannakis, Geert Leus
2020 arXiv   pre-print
We unveil the connections between Frank Wolfe (FW) type algorithms and the momentum in Accelerated Gradient Methods (AGM).  ...  In particular, we prove that a momentum variant of FW, that we term accelerated Frank Wolfe (AFW), converges with a faster rate Õ(1/k^2) on certain constraint sets despite the same O(1/k) rate as FW on  ...  Sinkhorn barycenters with free support via frank-wolfe algorithm. In Proc. Advances in Neural Info. Process. Syst., pages 9318-9329, 2019. Bin Shi, Simon S Du, Weijie J Su, and Michael I Jordan.  ... 
arXiv:2006.11116v1 fatcat:q4bxfjvq25fozmhwf5mxk4aglq

Generalized Self-Concordant Analysis of Frank-Wolfe algorithms [article]

Pavel Dvurechensky, Kamil Safin, Shimrit Shtern, Mathias Staudigl
2021 arXiv   pre-print
Projection-free optimization via different variants of the Frank-Wolfe (FW) method has become one of the cornerstones in large scale optimization for machine learning and computational statistics.  ...  Such generalized self-concordant (GSC) functions do not necessarily feature a Lipschitz continuous gradient, nor are they strongly convex.  ...  Acknowledgements The authors sincerely thank Professor Shoham Sabach for his contribution in the early stages of this project, including his part in developing the basic ideas developed in this paper.  ... 
arXiv:2010.01009v3 fatcat:k4nxnt6ez5ghril7bdpabp4h2u

Faster Rates for the Frank-Wolfe Method over Strongly-Convex Sets [article]

Dan Garber, Elad Hazan
2015 arXiv   pre-print
The Frank-Wolfe method (a.k.a. conditional gradient algorithm) for smooth optimization has regained much interest in recent years in the context of large scale optimization and machine learning.  ...  It is an active line of research to derive faster linear optimization-based algorithms for various settings of convex optimization.  ...  Introduction The Frank-Wolfe method, originally introduced by Frank and Wolfe in the 1950's (Frank & Wolfe, 1956) , is a first order method for the minimization of a smooth convex function over a convex  ... 
arXiv:1406.1305v2 fatcat:nitpfdmbnrde3pes63mwnycqae

kFW: A Frank-Wolfe style algorithm with stronger subproblem oracles [article]

Lijun Ding, Jicong Fan, Madeleine Udell
2021 arXiv   pre-print
This paper proposes a new variant of Frank-Wolfe (FW), called kFW.  ...  When the problem solution admits a sparse representation, both oracles are easy to compute, and kFW converges quickly for smooth convex objectives and several interesting constraint sets: kFW achieves  ...  Instead, researchers have turned to projection-free methods, such as the Frank-Wolfe algorithm (FW) [FW56] , also known as the conditional gradient method [LP66, Section 6].  ... 
arXiv:2006.16142v2 fatcat:drr3hwwp4vghbngx7oorw5lh2u

k-Means Clustering via the Frank-Wolfe Algorithm

Christian Bauckhage
2016 Lernen, Wissen, Daten, Analysen  
using the Frank-Wolfe algorithm.  ...  Seen from this point of view, k-means clustering can be computed using alternating least squares techniques and we show how the constrained optimization steps involved in this procedure can be solved efficiently  ...  For further details on the Frank-Wolfe algorithms as well as for a recent excellent survey of projection-free convex optimization over compact convex sets, we refer to [14] .  ... 
dblp:conf/lwa/Bauckhage16 fatcat:cm2e24wh5jg4xmnh7krl6iuxyu

Revisiting Projection-free Online Learning: the Strongly Convex Case [article]

Dan Garber, Ben Kretzu
2021 arXiv   pre-print
Projection-free optimization algorithms, which are mostly based on the classical Frank-Wolfe method, have gained significant interest in the machine learning community in recent years due to their ability  ...  This is somewhat surprising since it is known that for offline optimization, in general, strong convexity does not lead to faster rates for Frank-Wolfe.  ...  Revisiting frank-wolfe: Projectionfree sparse convex optimization. In ICML (1), pages 427-435, 2013. [19] Simon Lacoste-Julien and Martin Jaggi.  ... 
arXiv:2010.07572v2 fatcat:cytirb2tirantcndyddkpaw5pi

Linearly Convergent Frank-Wolfe with Backtracking Line-Search [article]

Fabian Pedregosa, Geoffrey Negiar, Armin Askari, Martin Jaggi
2020 arXiv   pre-print
Structured constraints in Machine Learning have recently brought the Frank-Wolfe (FW) family of algorithms back in the spotlight.  ...  However, these improved variants suffer from two practical limitations: they require at each iteration to solve a 1-dimensional minimization problem to set the step-size and also require the Frank-Wolfe  ...  Revisiting Frank-Wolfe: projection-free sparse convex optimization. In International Con- ference on Machine Learning, 2013. Thomas Kerdreux, Fabian Pedregosa, and Alexandre d'Aspremont.  ... 
arXiv:1806.05123v4 fatcat:tflor4uuwratrf4ibxfcngc5pm

On the Global Linear Convergence of Frank-Wolfe Optimization Variants [article]

Simon Lacoste-Julien, Martin Jaggi
2015 arXiv   pre-print
The Frank-Wolfe (FW) optimization algorithm has lately re-gained popularity thanks in particular to its ability to nicely handle the structured constraints appearing in machine learning applications.  ...  In this paper, we highlight and clarify several variants of the Frank-Wolfe optimization algorithm that have been successfully applied in practice: away-steps FW, pairwise FW, fully-corrective FW and Wolfe's  ...  One reason for the recent increased popularity of Frank-Wolfe-type algorithms is the sparsity of their iterates: in iteration t of the algorithm, the iterate can be represented as a sparse convex combination  ... 
arXiv:1511.05932v1 fatcat:2m664himyzgi3mvjoilsrsyt4a

Analysis of the Frank-Wolfe Method for Convex Composite Optimization involving a Logarithmically-Homogeneous Barrier [article]

Renbo Zhao, Robert M. Freund
2021 arXiv   pre-print
When specialized to the D-optimal design problem, we essentially recover the complexity obtained by Khachiyan using the Frank-Wolfe method with exact line-search.  ...  We present and analyze a new generalized Frank-Wolfe method for the composite optimization problem (P):min_x∈ℝ^n f(𝖠 x) + h(x), where f is a θ-logarithmically-homogeneous self-concordant barrier, 𝖠 is  ...  IEEE Transactions on Image Processing 21(3), 1084–1096 (2012) [34] Jaggi, M.: Revisiting Frank-Wolfe: Projection-free sparse convex optimization. In: Proc.  ... 
arXiv:2010.08999v4 fatcat:2n6sx4lf75fzdgmftfxj77l73y

Sequential Kernel Herding: Frank-Wolfe Optimization for Particle Filtering [article]

Simon Lacoste-Julien , Fredrik Lindsten, Francis Bach (LIENS, INRIA Paris - Rocquencourt, MSR - INRIA)
2015 arXiv   pre-print
In this paper, we propose to replace the random sampling step in a particle filter by Frank-Wolfe optimization.  ...  Recently, the Frank-Wolfe optimization algorithm was suggested as a procedure to obtain adaptive quadrature rules for integrals of functions in a reproducing kernel Hilbert space (RKHS) with a potentially  ...  This work was partially supported by the MSR-Inria Joint Centre, a grant by the European Research Council (SIERRA project 239993) and by the Swedish Research Council (project Learning of complex dynamical  ... 
arXiv:1501.02056v2 fatcat:zhfjneexa5favcetwmkjlfjjem

Revisiting Projection-Free Optimization for Strongly Convex Constraint Sets [article]

Jarrid Rector-Brooks, Jun-Kun Wang, Barzan Mozafari
2019 arXiv   pre-print
We revisit the Frank-Wolfe (FW) optimization under strongly convex constraint sets.  ...  strongly convex -- one of the fastest convergence rates in non-convex optimization.  ...  Frank-Wolfe (FW) optimization -In this paper, we focus on (FW) approaches, also known as projection-free or conditional gradient algorithms (Frank and Wolfe 1956) .  ... 
arXiv:1811.05831v3 fatcat:wx553mgzrnaadhdir23puuq5ni

Understanding Modern Techniques in Optimization: Frank-Wolfe, Nesterov's Momentum, and Polyak's Momentum [article]

Jun-Kun Wang
2021 arXiv   pre-print
Many existing optimization algorithms including Frank-Wolfe and Nesterov's acceleration methods can be recovered from the game by pitting two online learners with appropriate strategies against each other  ...  Moreover, we demonstrate that our approach of optimization as iteratively playing a game leads to three new fast Frank-Wolfe-like algorithms for some constraint sets, which further shows that our framework  ...  In this work, we propose a projection-free algorithm that enjoys a provably better convergence rate than the O(Lr 2 /T ) rate of Frank-Wolfe and Projected Gradient Descent (PGD) under some reasonable conditions  ... 
arXiv:2106.12923v1 fatcat:vfnqtcp3hrbunpqapw3n4tj6we

Differentially Private Matrix Completion Revisited [article]

Prateek Jain, Om Thakkar, Abhradeep Thakurta
2018 arXiv   pre-print
Our algorithm is based on the Frank-Wolfe method, and it consistently estimates the underlying preference matrix as long as the number of users m is ω(n^5/4), where n is the number of items, and each user  ...  while operating on sparse matrices.  ...  A Frank-Wolfe algorithm We use the classic Frank-Wolfe algorithm [22] as one of the optimization building blocks for our differentially private algorithms.  ... 
arXiv:1712.09765v2 fatcat:t7xc6nurxbcxdl357c5ytq3wtq
« Previous Showing results 1 — 15 out of 104 results