Filters








267 Hits in 3.1 sec

Block-Coordinate Frank-Wolfe Optimization for Structural SVMs [article]

Simon Lacoste-Julien, Martin Jaggi, Mark Schmidt, Patrick Pletscher
2013 arXiv   pre-print
We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints.  ...  However, unlike stochastic subgradient methods, the block-coordinate Frank-Wolfe algorithm allows us to compute the optimal step-size and yields a computable duality gap guarantee.  ...  Acknowledgements We thank Francis Bach, Bernd Gärtner and Ronny Luss for helpful discussions, and Robert Carnecky for the 3D illustration of Frank-Wolfe.  ... 
arXiv:1207.4747v4 fatcat:jbhv7draovfdvoun4bpfzoqsj4

Partial Linearization Based Optimization for Multi-class SVM [chapter]

Pritish Mohapatra, Puneet Kumar Dokania, C. V. Jawahar, M. Pawan Kumar
2016 Lecture Notes in Computer Science  
step-size in the descent direction that guarantees an increase in the dual objective, similar to Frank-Wolfe; and (iii) a block coordinate formulation similar to the one proposed for Frank-Wolfe, which  ...  We propose a novel partial linearization based approach for optimizing the multi-class svm learning problem.  ...  Fig. 2 : 2 Fig. 2: Comparison of Block-coordinate Frank-Wolfe (bcfw) and Block-coordinate Partial linearization (bcpl) in terms of the mean training time.  ... 
doi:10.1007/978-3-319-46454-1_51 fatcat:6i6zf5j35fc7riyf3ldxssv6ru

A multi-plane block-coordinate frank-wolfe algorithm for training structural SVMs with a costly max-oracle

Neel Shah, Vladimir Kolmogorov, Christoph H. Lampert
2015 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
The main idea is to (i) combine the recent stochastic Block-Coordinate Frank-Wolfe algorithm with efficient hyperplane caching, and (ii) use an automatic selection rule for deciding whether to call the  ...  Structural support vector machines (SSVMs) are amongst the best performing methods for structured computer vision tasks, such as semantic image segmentation or human pose estimation.  ...  Acknowledgements We thank Alexander Kolesnikov for his help with the segmentation datasets.  ... 
doi:10.1109/cvpr.2015.7298890 dblp:conf/cvpr/ShahKL15 fatcat:uwhxdwpbh5fadods3xmfnzfw4i

Minding the Gaps for Block Frank-Wolfe Optimization of Structured SVMs [article]

Anton Osokin, Jean-Baptiste Alayrac, Isabella Lukasewitz, Puneet K. Dokania, Simon Lacoste-Julien
2016 arXiv   pre-print
In this paper, we propose several improvements on the block-coordinate Frank-Wolfe (BCFW) algorithm from Lacoste-Julien et al. (2013) recently used to optimize the structured support vector machine (SSVM  ...  Second, we incorporate pairwise and away-step variants of Frank-Wolfe into the block-coordinate setting. Third, we cache oracle calls with a cache-hit criterion based on the block gaps.  ...  Block-coordinate Frank-Wolfe optimization for structural SVMs. In Proceedings of the International Conference on Machine Learning (ICML), 2013. Mitchell, B., Demyanov, V. F., and Malozemov, V.  ... 
arXiv:1605.09346v1 fatcat:s3wetk6lc5d63f7ugn6yeesbvq

A Multi-Plane Block-Coordinate Frank-Wolfe Algorithm for Training Structural SVMs with a Costly max-Oracle [article]

Neel Shah, Vladimir Kolmogorov, Christoph H. Lampert
2014 arXiv   pre-print
The main idea is to (i) combine the recent stochastic Block-Coordinate Frank-Wolfe algorithm with efficient hyperplane caching, and (ii) use an automatic selection rule for deciding whether to call the  ...  Structural support vector machines (SSVMs) are amongst the best performing models for structured computer vision tasks, such as semantic image segmentation or human pose estimation.  ...  Block-coordinate Frank-Wolfe algorithm The block-coordinate Frank-Wolfe algorithm [18] also solves the dual of problem (1), but it improves over the Algorithm 1 Frank-Wolfe algorithm for the dual of  ... 
arXiv:1408.6804v2 fatcat:rojxs3bk5vfobkqq2jqrdcoeou

Parallel and Distributed Block-Coordinate Frank-Wolfe Algorithms [article]

Yu-Xiang Wang and Veeranjaneyulu Sadhanala and Wei Dai and Willie Neiswanger and Suvrit Sra and Eric P. Xing
2016 arXiv   pre-print
Our algorithms assume block-separable constraints, and subsume the recent Block-Coordinate Frank-Wolfe (BCFW) method lacoste2013block.  ...  We present experiments on structural SVM and Group Fused Lasso, obtaining significant speedups over competing state-of-the-art (and synchronous) methods.  ...  The Block-Coordinate Frank-Wolfe update for the i-th block maybe written as α k+1 (i) = α k i + γ(s i − α k (i) ) where γ is the step-size.  ... 
arXiv:1409.6086v2 fatcat:qtf4bgwoqnawlhg3xyuivhixe4

Primal-Dual Block Generalized Frank-Wolfe

Qi Lei, Jiacheng Zhuo, Constantine Caramanis, Inderjit S. Dhillon, Alexandros G. Dimakis
2019 Neural Information Processing Systems  
We propose a generalized variant of Frank-Wolfe algorithm for solving a class of sparse/low-rank optimization problems.  ...  The proposed Primal-Dual Block Generalized Frank-Wolfe algorithm reduces the per-iteration cost while maintaining linear convergence rate.  ...  We propose a generalized variant of FW that we call Primal-Dual Block Generalized Frank Wolfe.  ... 
dblp:conf/nips/LeiZCDD19 fatcat:ywlxk5hhvzgdlhmr4mcgfox3zq

Efficient Training of Structured SVMs via Soft Constraints

Ofer Meshi, Nathan Srebro, Tamir Hazan
2015 International Conference on Artificial Intelligence and Statistics  
Our method, based on the Frank-Wolfe algorithm, achieves significant speedups over existing state-of-theart methods without hurting prediction accuracy.  ...  Structured output prediction is a powerful framework for jointly predicting interdependent output labels.  ...  Algorithm 1 1 Block-coordinate Frank-Wolfe for soft structured SVM 1: Initialize: w = 0, = 0, µ (m) ↵ (y ↵ ) = {y ↵ = y (m) ↵ } for all m, ↵, y ↵ 2: while not converged do Figure 1 : 1 Figure 1: (Left  ... 
dblp:conf/aistats/MeshiSH15 fatcat:bvktqsanxzgc7m633ia2ix6wlm

Semi-Stochastic Frank-Wolfe Algorithms with Away-Steps for Block-Coordinate Structure Problems [article]

Donald Goldfarb, Garud Iyengar, Chaoxu Zhou
2016 arXiv   pre-print
We propose a semi-stochastic Frank-Wolfe algorithm with away-steps for regularized empirical risk minimization and extend it to problems with block-coordinate structure.  ...  In preliminary numerical tests on structural SVM and graph-guided fused LASSO, our algorithms outperform other competing algorithms in both iteration cost and total number of data passes.  ...  With above notation, we can apply the stochastic block coordinate Frank-Wolfe algorithm with away-steps the the structural SVM problem.  ... 
arXiv:1602.01543v3 fatcat:oacfl4wivrh3naoij3qlzaspye

Large-Scale Multiclass Support Vector Machine Training via Euclidean Projection onto the Simplex

Mathieu Blondel, Akinori Fujino, Naonori Ueda
2014 2014 22nd International Conference on Pattern Recognition  
Dual decomposition methods are the current stateof-the-art for training multiclass formulations of Support Vector Machines (SVMs).  ...  At every iteration, dual decomposition methods update a small subset of dual variables by solving a restricted optimization problem.  ...  [6] , used Sequential Minimal Optimization (SMO) [11] to obtain an approximate solution. More recently, Lacoste-Julien et al. [12] proposed a block Frank-Wolfe method.  ... 
doi:10.1109/icpr.2014.231 dblp:conf/icpr/BlondelFU14 fatcat:2cm5n5fdrvdm7jkf6rykbxc3zq

Learning Weighted Top-k Support Vector Machine

Yoshihiro Hirohashi, Tsuyoshi Kato
2020 Journal of Information Processing  
We developed a new optimization algorithm based on the Frank-Wolfe algorithm that requires no step size, enjoys the clear stopping criterion, and is never solicitous for computational instability.  ...  The Frank-Wolfe algorithm repeats the direction finding step and the line search step. The discoveries in this study are that both the steps can be given in a closed form.  ...  At each iteration, one of the blocks is chosen randomly, and the rest of the blocks are fixed whereas the chosen block is optimized. Lapin et al. [10] have employed SDCA to train the top-k SVM.  ... 
doi:10.2197/ipsjjip.28.387 fatcat:eq632dsrivfopozm3som35lji4

Primal-Dual Block Frank-Wolfe [article]

Qi Lei, Jiacheng Zhuo, Constantine Caramanis, Inderjit S. Dhillon, Alexandros G. Dimakis
2019 arXiv   pre-print
We propose a variant of the Frank-Wolfe algorithm for solving a class of sparse/low-rank optimization problems.  ...  The proposed Primal-Dual Block Frank-Wolfe algorithm reduces the per-iteration cost while maintaining linear convergence rate.  ...  In this paper we tackle the challenges by exploiting the special structure induced by the constraints and FW steps. We propose a variant of FW that we call Primal-Dual Block Frank Wolfe.  ... 
arXiv:1906.02436v1 fatcat:zcfxseqt3zewxfj2rqolpowcsa

Dual Decomposed Learning with Factorwise Oracle for Structural SVM of Large Output Domain

Ian En-Hsu Yen, Xiangru Huang, Kai Zhong, Ruohan Zhang, Pradeep Ravikumar, Inderjit S. Dhillon
2016 Neural Information Processing Systems  
Structural SVMs.  ...  In this work, we show that by decomposing training of a Structural Support Vector Machine (SVM) into a series of multiclass SVM problems connected through messages, one can replace an expensive structured  ...  In practice, a Block-Coordinate Frank-Wolfe (BCFW) method has much faster convergence than Frank-Wolfe method (Algorithm 2) [13, 9] , but proving linear convergence for BCFW is also much more difficult  ... 
dblp:conf/nips/YenHZZRD16 fatcat:3gt4r5ap4nchphx27eqgeilz54

PD-Sparse : A Primal and Dual Sparse Approach to Extreme Multiclass and Multilabel Classification

Ian En-Hsu Yen, Xiangru Huang, Pradeep Ravikumar, Kai Zhong, Inderjit S. Dhillon
2016 International Conference on Machine Learning  
We thus propose a Fully-Corrective Block-Coordinate Frank-Wolfe (FC-BCFW) algorithm that exploits both primal and dual sparsity to achieve a complexity sublinear to the number of primal and dual variables  ...  However, as the diversity of labels increases in the feature space, structural assumption can be easily violated, which leads to degrade in the testing performance.  ...  We thus propose a Fully-Corrective Block Coordinate Frank-Wolfe algorithm to solve the primal-dual sparse problem given by margin-maximizing loss with `1-`2 penalties.  ... 
dblp:conf/icml/YenHRZD16 fatcat:lgu72gz6ezepfij3hdgqo4rms4

Learning from Video and Text via Large-Scale Discriminative Clustering [article]

Antoine Miech, Jean-Baptiste Alayrac, Piotr Bojanowski, Ivan Laptev, Josef Sivic
2017 arXiv   pre-print
We address this issue and propose an online optimization algorithm based on the Block-Coordinate Frank-Wolfe algorithm.  ...  Conclusion We have proposed an efficient online optimization method based on the Block-Coordinate Frank-Wolfe algorithm.  ...  A variant of Frank-Wolfe with randomized block coordinate descent was proposed in [21] .  ... 
arXiv:1707.09074v1 fatcat:bxup4vtk5nh6xd7zw2vnrog6je
« Previous Showing results 1 — 15 out of 267 results