62,377 Hits in 1.9 sec

Non-convex Optimization for Machine Learning

Prateek Jain, Purushottam Kar
2017 Foundations and Trends® in Machine Learning  
A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems.  ...  The freedom to express the learning problem as a non-convex optimization problem gives immense modeling power to the algorithm designer, but often such problems are NP-hard to solve.  ...  As such, this monograph can be used for a semester-length course on the basics of non-convex optimization with applications to machine learning.  ... 
doi:10.1561/2200000058 fatcat:auzksnxgcbgfrkcdi2iwrx4lq4

Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study [article]

Peng Xu, Farbod Roosta-Khorasani, Michael W. Mahoney
2018 arXiv   pre-print
non-convex ML problems.  ...  While first-order optimization methods such as stochastic gradient descent (SGD) are popular in machine learning (ML), they come with well-known deficiencies, including relatively-slow convergence, sensitivity  ...  Amir Gholaminejad for valuable comments on our empirical evaluations and suggestions for improving them.  ... 
arXiv:1708.07827v2 fatcat:mrug5vs5x5ac7e7bbq7fthtexe

Non-negative matrix completion for bandwidth extension: A convex optimization approach

Dennis L. Sun, Rahul Mazumder
2013 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP)  
One approach that has been shown to be successful for bandwidth extension is non-negative matrix factorization (NMF).  ...  We formulate bandwidth extension as a convex optimization problem, propose a simple algorithm, and demonstrate the effectiveness of this approach on practical examples.  ...  For β ∈ [1, 2], d β is convex in its second argument, so the non-convexity in (2) arises only from the rank constraint.  ... 
doi:10.1109/mlsp.2013.6661924 dblp:conf/mlsp/SunM13 fatcat:dbiq26cgnze6tky2hszzovrwna

Non-asymptotic convergence analysis of inexact gradient methods for machine learning without strong convexity

Anthony Man-Cho So, Zirui Zhou
2017 Optimization Methods and Software  
Many recent applications in machine learning and data fitting call for the algorithmic solution of structured smooth convex optimization problems.  ...  of IGMs when applied to a class of structured convex optimization problems.  ...  Introduction Motivated by various applications in machine learning and data fitting, there has been much interest in the design and analysis of fast algorithms for solving large-scale structured convex  ... 
doi:10.1080/10556788.2017.1296439 fatcat:tpmk5tamlzbgjimwiyvdozuig4

Preface: Special issue on learning and robustness

Panos M. Pardalos
2013 Computational Management Science  
In the paper "Numerical Study of Learning Algorithms on Stiefel Manifold" by Takafumi Kanamori and Akiko Takeda non-convex optimization problems in machine learning are presented.  ...  In this special issue of CMS the authors have considered several issues of machine learning and non-convex optimization algorithms with several applications in finance.  ...  In the paper "Numerical Study of Learning Algorithms on Stiefel Manifold" by Takafumi Kanamori and Akiko Takeda non-convex optimization problems in machine learning are presented.  ... 
doi:10.1007/s10287-013-0187-1 fatcat:5ayvjhu37fezflyu2gneakclxa

A Comparison of First-order Algorithms for Machine Learning [article]

Yu Wei, Pock Thomas
2014 arXiv   pre-print
In this work, we demonstrate a comprehensive comparison of some state-of-the-art first-order optimization algorithms for convex optimization problems in machine learning.  ...  We concentrate on several smooth and non-smooth machine learning problems with a loss function plus a regularizer.  ...  Introduction Optimization is the key of machine learning. Most machine learning problems can be cast as optimization problems.  ... 
arXiv:1404.6674v1 fatcat:mcnisq5ptva2bcbgoc32dbdtxm

Grand Challenges in Signal Processing for Communications

Changyang She, Peng Cheng, Ang Li, Yonghui Li
2021 Frontiers in Signal Processing  
surfaces, multi-user signal processing, machine learning, MmWave and THz communications, spectrum and energy efficient communications, cross-layer optimization and convex and non-convex optimization.  ...  It is worth noting that by solving the KKT conditions, one can only obtain a local optimal solution for non-convex problems. Only for convex problems, a local optimal solution is also global optimal.  ... 
doi:10.3389/frsip.2021.664331 fatcat:rjc4f7ffjnhinbxhyb34frpmpy

Conditions for Convergence in Regularized Machine Learning Objectives [article]

Patrick Hop, Xinghao Pan
2013 arXiv   pre-print
This paper will serve as a helpful cheat-sheet for machine learning practitioners encountering this problem class in the field.  ...  Analysis of the convergence rates of modern convex optimization algorithms can be achived through binary means: analysis of emperical convergence, or analysis of theoretical convergence.  ...  objective seperability is exploited, and the computation is distributed over n-machines.  ... 
arXiv:1305.4081v1 fatcat:mbxzvjqvyrfklbx3ikrknf6sji

GS-OPT: A new fast stochastic algorithm for solving the non-convex optimization problem

Xuan Bui, Nhung Duong, Trung Hoang
2020 IAES International Journal of Artificial Intelligence (IJ-AI)  
<p>Non-convex optimization has an important role in machine learning. However, the theoretical understanding of non-convex optimization remained rather limited.  ...  In this paper, we have proposed a new algorithm namely GS-OPT (General Stochastic OPTimization) which is effective for solving the non-convex problems.  ...  INTRODUCTION In machine learning, there are a lot of problems that lead to non-convex optimization.  ... 
doi:10.11591/ijai.v9.i2.pp183-192 fatcat:wr5kbi2vi5gxjfqfzsvfdatjli

Stochastic Structured Prediction under Bandit Feedback [article]

Artem Sokolov and Julia Kreutzer and Christopher Lo and Stefan Riezler
2016 arXiv   pre-print
Best results under both criteria are obtained for a non-convex objective for pairwise preference learning under bandit feedback.  ...  We present applications of this learning scenario to convex and non-convex objectives for structured prediction and analyze them as stochastic first-order methods.  ...  Lastly, we analyze the pairwise preference learning algorithm introduced by [19] . This algorithm can also be analyzed as an SFO method for non-convex optimization.  ... 
arXiv:1606.00739v2 fatcat:khddswq23vcy7esrn2rcbb2tmi

Support Vector Machine optimization with fractional gradient descent for data classification

Dian Puspita Hapsari, Imam Utoyo, Santi Wulan Purnami
2021 Journal of Applied Sciences, Management and Engineering Technology  
The Fractional gradient descent method is an unconstrained optimization algorithm to train classifiers with support vector machines that have convex problems.  ...  SVM is a reliable linear classifier for linear or non-linear data, for large-scale data, there are computational time constraints.  ...  Gratitude's for the supervisor of the doctoral program, the faculty of science and technology, Airlangga University.  ... 
doi:10.31284/j.jasmet.2021.v2i1.1467 fatcat:2ikclowkabfjlekavr7fco6nqi

Online Compact Convexified Factorization Machine [article]

Wenpeng Zhang, Xiao Lin, Peilin Zhao
2018 arXiv   pre-print
The initial challenge is that no prior formulations of FM could fulfill the requirements in Online Convex Optimization (OCO) -- the paramount framework for online learning algorithm design.  ...  Factorization Machine (FM) is a supervised learning approach with a powerful capability of feature engineering.  ...  Online Convex Optimization Online Convex Optimization (OCO) [12, 26] is the paramount framework for designing online learning algorithms.  ... 
arXiv:1802.01379v1 fatcat:pu3bcebbrvem3oohnxptvopnhu

Convex Optimization: Algorithms and Complexity

Sébastien Bubeck
2015 Foundations and Trends® in Machine Learning  
Some convex optimization problems in machine learning Many fundamental convex optimization problems in machine learning take the following form: min. x∈R n m i=1 f i (x) + λR(x), (1.1) where the functions  ...  This is especially interesting for stochastic optimization, and very relevant to machine learning applications.  ... 
doi:10.1561/2200000050 fatcat:dtfrxqmrpratnnb7iw7ce5erg4

Theoretical Limits of Pipeline Parallel Optimization and Application to Distributed Deep Learning [article]

Igor Colin, Ludovic Dos Santos, Kevin Scaman
2019 arXiv   pre-print
Finally, we perform an empirical analysis of the non-smooth non-convex case and show that, for difficult and highly non-smooth problems, PPRS outperforms more traditional optimization algorithms such as  ...  For non-smooth convex functions, we provide a novel algorithm coined Pipeline Parallel Random Smoothing (PPRS) that is within a d^1/4 multiplicative factor of the optimal convergence rate, where d is the  ...  Unfortunately, results about randomized sampling for non-convex problems [15, 16] are ill-suited for machine learning scenarios: linesearch is at the core of the method, requiring prohibitive evaluations  ... 
arXiv:1910.05104v1 fatcat:7c5j65h46rhl7pt3eokfwy42gm

A Unified Robust Classification Model [article]

Akiko Takeda , Takafumi Kanamori
2012 arXiv   pre-print
We give a statistical interpretation of the unified classification model and propose a non-convex optimization algorithm that can be applied to non-convex variants of existing learning methods.  ...  A wide variety of machine learning algorithms such as support vector machine (SVM), minimax probability machine (MPM), and Fisher discriminant analysis (FDA), exist for binary classification.  ...  For 0 ∈ int(U), RCM (2) is essentially a non-convex problem, and we need to use non-convex optimization methods to solve it.  ... 
arXiv:1206.4599v1 fatcat:dj6rvukwxnecdjyipy7bcuswzq
« Previous Showing results 1 — 15 out of 62,377 results