Filters








4,536 Hits in 5.6 sec

Efficient Projection-Free Online Methods with Stochastic Recursive Gradient

Jiahao Xie, Zebang Shen, Chao Zhang, Boyu Wang, Hui Qian
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
This paper focuses on projection-free methods for solving smooth Online Convex Optimization (OCO) problems.  ...  To fill this gap, two efficient projection-free online methods called ORGFW and MORGFW are proposed for solving stochastic and adversarial OCO problems, respectively.  ...  Online Stochastic Recursive Gradient-Based Frank-Wolfe In this section, we present our projection-free methods for solving OCO problems.  ... 
doi:10.1609/aaai.v34i04.6116 fatcat:jgv2k6su5zafnkosptnhkxxbhq

Efficient Projection-Free Online Methods with Stochastic Recursive Gradient [article]

Jiahao Xie, Zebang Shen, Chao Zhang, Boyu Wang, Hui Qian
2019 arXiv   pre-print
This paper focuses on projection-free methods for solving smooth Online Convex Optimization (OCO) problems.  ...  To fill this gap, two efficient projection-free online methods called ORGFW and MORGFW are proposed for solving stochastic and adversarial OCO problems, respectively.  ...  Online Stochastic Recursive Gradient-Based Frank-Wolfe In this section, we present our projection-free methods for solving OCO problems.  ... 
arXiv:1910.09396v2 fatcat:ygt457pl25e7jm47mud5jvlpx4

Fully Projection-free Proximal Stochastic Gradient Method with Optimal Convergence Rates

Yan Li, Xiaofeng Cao, Honghui Chen
2020 IEEE Access  
Motivated by this problem, we propose a fully projection-free proximal stochastic gradient method, which has two advantages over previous methods. First, it enjoys high efficiency.  ...  Proximal stochastic gradient plays an important role in large-scale machine learning and big data analysis. It needs to iteratively update models within a feasible set until convergence.  ...  FULLY PROJECTION-FREE PROXIMAL STOCHASTIC GRADIENT In this section, to efficiently solve the large-scale optimization problem, we propose a fully projection-free proximal stochastic gradient (FAP) method  ... 
doi:10.1109/access.2020.3019885 fatcat:biauxqkujral7es2yws2hjp7ou

Parametric value function approximation: A unified view

Matthieu Geist, Olivier Pietquin
2011 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL)  
Related algorithms are derived by considering one of the associated cost functions and a specific way to minimize it, almost always a stochastic gradient descent or a recursive least-squares approach.  ...  This survey reviews and unifies state of the art methods for parametric value function approximation by grouping them into three main categories: bootstrapping, residuals and projected fixed-point approaches  ...  Such an approach as a clear advantage for methods based on stochastic gradient descent, as it speeds up the learning.  ... 
doi:10.1109/adprl.2011.5967355 dblp:conf/adprl/GeistP11 fatcat:hhntw5pucvafxcmxknr6da57ve

Projection-Free Online Optimization with Stochastic Gradient: From Convexity to Submodularity [article]

Lin Chen, Christopher Harshaw, Hamed Hassani, Amin Karbasi
2018 arXiv   pre-print
In this work, we propose Meta-Frank-Wolfe, the first online projection-free algorithm that uses stochastic gradient estimates.  ...  Current methods for online convex optimization require either a projection or exact gradient computation at each step, both of which can be prohibitively expensive for large-scale applications.  ...  Hazan & Luo (2016) devised a projection-free stochastic convex optimization algorithm based on this technique.  ... 
arXiv:1802.08183v4 fatcat:avij3ca6i5ae7b67gezaoa6ulu

Algorithmic Survey of Parametric Value Function Approximation

M. Geist, O. Pietquin
2013 IEEE Transactions on Neural Networks and Learning Systems  
Related algorithms are derived by considering one of the associated cost functions and a specific minimization method, generally a stochastic gradient descent or a recursive least-squares approach.  ...  This survey reviews state-of-the-art methods for (parametric) value function approximation by grouping them into three main categories: bootstrapping, residual and projected fixed-point approaches.  ...  This research was partly funded by the EU FP7 FET project ILHAIRE (grant n • 270780) and the Région Lorraine (France).  ... 
doi:10.1109/tnnls.2013.2247418 pmid:24808468 fatcat:liwiujjbufaijcpge3hzii5234

Time‐varying parameter estimation with application to trajectory tracking

K. Bousson
2007 Aircraft Engineering and Aerospace Technology  
Practical implications -The proposed NLRA method may be adopted for recursive parameter estimation of uncertain systems when no stochastic information is available.  ...  Purpose -This paper is concerned with an online parameter estimation algorithm for nonlinear uncertain time-varying systems for which no stochastic information is available.  ...  Conclusion A method for online parameter estimation is presented for nonlinear uncertain systems for which no stochastic information is available.  ... 
doi:10.1108/00022660710758277 fatcat:apoelb2e55cwnmpgnqc7v6hfji

Online Principal Component Analysis in High Dimension: Which Algorithm to Choose? [article]

Hervé Cardot, David Degras
2015 arXiv   pre-print
This work provides guidance for selecting an online PCA algorithm in practice.  ...  We present the main approaches to online PCA, namely, perturbation techniques, incremental methods, and stochastic optimization, and compare their statistical accuracy, computation time, and memory requirements  ...  Stochastic approximation Stochastic gradient optimization Stochastic gradient approaches adopt a rather different point of view based on the population version of the optimization problem (1).  ... 
arXiv:1511.03688v1 fatcat:jb4aalwlovcjnekimtauncpta4

Accelerated Stochastic Gradient-free and Projection-free Methods [article]

Feihu Huang, Lue Tao, Songcan Chen
2020 arXiv   pre-print
In the paper, we propose a class of accelerated stochastic gradient-free and projection-free (a.k.a., zeroth-order Frank-Wolfe) methods to solve the constrained stochastic and finite-sum nonconvex optimization  ...  Specifically, we propose an accelerated stochastic zeroth-order Frank-Wolfe (Acc-SZOFW) method based on the variance reduced technique of SPIDER/SpiderBoost and a novel momentum accelerated technique.  ...  Acknowledgements We thank the anonymous reviewers for their valuable comments.  ... 
arXiv:2007.12625v2 fatcat:ld6aps74dvcf5hwulaovd27pii

An Online Prediction Algorithm for Reinforcement Learning with Linear Function Approximation using Cross Entropy Method [article]

Ajin George Joseph, Shalabh Bhatnagar
2018 arXiv   pre-print
In this paper, we provide two new stable online algorithms for the problem of prediction in reinforcement learning, i.e., estimating the value function of a model-free Markov reward process using the linear  ...  The algorithms employ the multi-timescale stochastic approximation variant of the very popular cross entropy (CE) optimization method which is a model based search method to find the global optimum of  ...  Other methods in this class are model reference adaptive search (MRAS) [17] , gradient-based adaptive stochastic search for simulation optimization (GASSO) [54] , ant colony optimization (ACO) [14]  ... 
arXiv:1806.06720v1 fatcat:23reo76utngipbqjyshxzkrg6i

An online prediction algorithm for reinforcement learning with linear function approximation using cross entropy method

Ajin George Joseph, Shalabh Bhatnagar
2018 Machine Learning  
In this paper, we provide two new stable online algorithms for the problem of prediction in reinforcement learning, i.e., estimating the value function of a model-free Markov reward process using the linear  ...  The algorithms employ the multi-timescale stochastic approximation variant of the very popular cross entropy optimization method which is a model based search method to find the global optimum of a real-valued  ...  In this paper, we apply a gradient-free technique called the cross entropy (CE) method instead to find the minimum.  ... 
doi:10.1007/s10994-018-5727-z fatcat:y7xsxpiqtjbr7ofydnthxypfsu

A Survey of Optimization Methods from a Machine Learning Perspective [article]

Shiliang Sun, Zehui Cao, Han Zhu, Jing Zhao
2019 arXiv   pre-print
The systematic retrospect and summary of the optimization methods from the perspective of machine learning are of great significance, which can offer guidance for both developments of optimization and  ...  Finally, we explore and give some challenges and open problems for the optimization in machine learning.  ...  Meanwhile, it overcomes the disadvantage of batch gradient descent that cannot be used for online learning.  ... 
arXiv:1906.06821v2 fatcat:rcaas4ccpbdffhuvzcg2oryxr4

Projection-free Distributed Online Learning in Networks

Wenpeng Zhang, Peilin Zhao, Wenwu Zhu, Steven C. H. Hoi, Tong Zhang
2017 International Conference on Machine Learning  
In this paper, we fill this gap by proposing the distributed online conditional gradient algorithm, which eschews the expensive projection operation needed in its counterpart algorithms by exploiting much  ...  However, none of existing studies has explored it in the distributed online learning setting, where locally light computation is assumed.  ...  Acknowledgements This work is supported by National Program on Key Basic Research Project No. 2015CB352300 and National Natural Science Foundation of China Major Project No. U1611461.  ... 
dblp:conf/icml/ZhangZZHZ17 fatcat:ocwddmymgbb3fl45nnmyszfnc4

Exploiting Smoothness in Statistical Learning, Sequential Prediction, and Stochastic Optimization [article]

Mehrdad Mahdavi
2014 arXiv   pre-print
The overarching goal of this thesis is to reassess the smoothness of loss functions in statistical learning, sequential prediction/online learning, and stochastic optimization and explicate its consequences  ...  In the last several years, the intimate connection between convex optimization and learning problems, in both statistical and sequential frameworks, has shifted the focus of algorithmic machine learning  ...  Another problem we address in this thesis is efficient projection-free optimization methods for stochastic and online convex optimization.  ... 
arXiv:1407.5908v1 fatcat:vlevdkb23bfibombrkqttvtlp4

A Cross Entropy based Stochastic Approximation Algorithm for Reinforcement Learning with Linear Function Approximation [article]

Ajin George Joseph, Shalabh Bhatnagar
2016 arXiv   pre-print
This is the first time a model based search method is used for the prediction problem. The application of CE to a stochastic setting is a completely unexplored domain.  ...  In this paper, we provide a new algorithm for the problem of prediction in Reinforcement Learning, i.e., estimating the Value Function of a Markov Reward Process (MRP) using the linear function approximation  ...  In this paper, we apply a gradient-free technique called the Cross Entropy (CE) method instead to find the minimum.  ... 
arXiv:1609.09449v1 fatcat:igowd5qtznfvbbispfzzrf4u4e
« Previous Showing results 1 — 15 out of 4,536 results