6,351 Hits in 3.7 sec

Deep Reinforcement Learning via L-BFGS Optimization [article]

Jacob Rafati, Roummel F. Marcia
2019 arXiv   pre-print
The limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) approach is one of the most popular quasi-Newton methods that construct positive definite Hessian approximations.  ...  In this paper, we introduce an efficient optimization method, based on the limited memory BFGS quasi-Newton method using line search strategy -- as an alternative to SGD methods.  ...  By measuring and storing the difference between consecutive gradients, quasi-Newton methods construct quasi-Newton matrices, {B k }, which are low-rank updates to the previous Hessian approximations for  ... 
arXiv:1811.02693v2 fatcat:dgqrwcko5vbmddysmhch5fg244

A quasi-Newton Algorithm on the Orthogonal Manifold for NMF with Transform Learning

Pierre Ablin, Dylan Fagot, Herwig Wendt, Alexandre Gramfort, Cedric Fevotte
2019 ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)  
In this paper, we derive a quasi-Newton method on the manifold using sparse approximations of the Hessian.  ...  When assumed orthogonal (like the Fourier or Cosine transforms), learning the transform yields a nonconvex optimization problem on the orthogonal matrix manifold.  ...  The three different transform learning algorithms are run with L = 5. Results for the two experiments are shown in Fig. 2 and illustrate the superiority of the proposed quasi-Newton algorithm.  ... 
doi:10.1109/icassp.2019.8683291 dblp:conf/icassp/AblinFWGF19 fatcat:z5h4nucvy5fbvi6e3cfvjbau6y

A Nesterov's accelerated quasi-Newton method for global routing using deep reinforcement learning

Indrapriyadarsini S, Shahrzad Mahboubi, Hiroshi Ninomiya, Takeshi Kamio, Hideki Asai
2021 Nonlinear Theory and Its Applications IEICE  
This paper attempts to accelerate the training of deep Q-networks by introducing a second order Nesterov's accelerated quasi-Newton method and verify the feasibility of second order methods in deep reinforcement  ...  learning.  ...  The Q-learning method, a form of temporal difference learning, uses the Q-value function that satisfies the Bellman equation to maximize the cumulative reward.  ... 
doi:10.1587/nolta.12.323 fatcat:if4olaiuxvdg5kdz2p3mdksk5e

A Quasi-Newton algorithm on the orthogonal manifold for NMF with transform learning [article]

Pierre Ablin , Alexandre Gramfort
2018 arXiv   pre-print
In this paper, we derive a quasi-Newton method on the manifold using sparse approximations of the Hessian.  ...  When assumed orthogonal (like the Fourier or Cosine transforms), learning the transform yields a non-convex optimization problem on the orthogonal matrix manifold.  ...  The three different transform learning algorithms are run with L = 5. Results for the two experiments are shown in Fig. 2 and illustrate the superiority of the proposed quasi-Newton algorithm.  ... 
arXiv:1811.02225v1 fatcat:r2tjpq7k4jh2fdhxj4apvb355a

Nonmonotone BFGS-trained recurrent neural networks for temporal sequence processing

Chun-Cheng Peng, George D. Magoulas
2011 Applied Mathematics and Computation  
In this paper we propose a nonmonotone approach to recurrent neural networks training for temporal sequence processing applications.  ...  This approach allows learning performance to deteriorate in some iterations, nevertheless the network's performance is improved over time.  ...  Methods that satisfy the quasi-Newton condition, i.e., B kþ1ŝk ¼ŷ k , whereŝ k ¼ w kþ1 À w k , andŷ k ¼ g kþ1 À g k , can be considered as members of the class of quasi-Newton methods.  ... 
doi:10.1016/j.amc.2010.12.012 fatcat:neps6mha25bbhetgt5tliy6jmq

Quasi-Newton Iteration in Deterministic Policy Gradient [article]

Arash Bahari Kordabad, Hossein Nejatbakhsh Esfahani, Wenqi Cai, Sebastien Gros
2022 arXiv   pre-print
This paper presents a model-free approximation for the Hessian of the performance of deterministic policies to use in the context of Reinforcement Learning based on Quasi-Newton steps in the policy parameters  ...  We show that the approximate Hessian converges to the exact Hessian at the optimal policy, and allows for a superlinear convergence in the learning, provided that the policy parametrization is rich.  ...  A Quasi-Newton method is developed in [11] for Temporal Difference (TD) learning in order to get faster convergence. Natural Actor-critic has been investigated in [12] .  ... 
arXiv:2203.13854v1 fatcat:nyp2ca3rtneq7l7mtbrfc3o6d4

A CCA Criterion Based Adaptive Algorithm for Blind Extraction of Specific Signal

Wei-Tao Zhang
2014 International Journal of Information and Electronics Engineering  
The experiment results demonstrate the effectiveness of the proposed learning algorithms, and show that the modified Newton algorithm converges much faster than the other extraction method.  ...  The first algorithm is based on the steepest descent technique, and the second one is a modified Newton algorithm.  ...  This will lead to different behavior of learning algorithms. We also proposed the steepest descent learning and modified Newton learning algorithms to optimize the proposed criterion.  ... 
doi:10.7763/ijiee.2014.v4.466 fatcat:kkmo3qf4rbhgzm4qnfwe54e2mm

Combining system identification with reinforcement learning-based MPC [article]

Andreas B. Martinsen, Anastasios M. Lekkas, Sebastien Gros
2020 arXiv   pre-print
In this paper we propose and compare methods for combining system identification (SYSID) and reinforcement learning (RL) in the context of data-driven model predictive control (MPC).  ...  Assuming a known model structure of the controlled system, and considering a parametric MPC, the proposed approach simultaneously: a) Learns the parameters of the MPC using RL in order to optimize performance  ...  For faster learning, we propose using a second order approach, and perform quasi-Newton steps on the parameters.  ... 
arXiv:2004.03265v1 fatcat:23ppmfl7szaq3ldcfvlimqdh3e

Understanding the Principles of Recursive Neural networks: A Generative Approach to Tackle Model Complexity [article]

Alejandro Chinea
2009 arXiv   pre-print
Secondly, we propose an approximate second order stochastic learning algorithm.  ...  In particular, not only for being extremely complex information processing models, but also because of a computational expensive learning phase.  ...  Therefore, taking into account that the proposed learning rule behaves like a stochastic approximation of a quasi-Newton method, the proposed algorithm achieves a good trade-off in terms of memory storage  ... 
arXiv:0911.3298v1 fatcat:35hrog6zujbmtkamgwj3rmob5y

Improved modeling of lithium-ion battery capacity degradation using an individual-state training method and recurrent softplus neural network

Jianxiang Wang, Xuesong Feng, Xiaokun Zhang, Yong Xiang
2020 IEEE Access  
The training method is based on the Broyden-Fletcher-Goldfarb-Shanno quasi-Newton method, and is modified to adapt to different battery samples by training the initial states for individual batteries to  ...  This is equivalent to training a different model for each battery with shared model parameters, which improves the generalization of the model while preserving the model's capability to account for individual  ...  Based on the BFGS quasi-Newton method, the individual-state training method utilizes training data from multiple degradation tests and adapts to different battery data by training the initial VOLUME 9  ... 
doi:10.1109/access.2020.3048146 fatcat:xik4hq6itfcqres6noffslseeu

Improved Quasi-Newton Adaptive-Filtering Algorithm

Md Zulfiquar Ali Bhotto, Andreas Antoniou
2010 IEEE Transactions on Circuits and Systems Part 1: Regular Papers  
Index Terms-Adaptation algorithms, adaptive filters, convergence speed in adaptation algorithms, quasi-Newton algorithms, steady-state misalignment.  ...  An improved quasi-Newton (QN) algorithm that performs data-selective adaptation is proposed whereby the weight vector and the inverse of the input-signal autocorrelation matrix are updated only when the  ...  The known quasi-Newton (KQN) algorithm reported in [2] , [3] offers better numerical robustness whereas the LMS-Newton (LMSN) algo-Manuscript received April 14, 2009 rithms reported in [4] offer  ... 
doi:10.1109/tcsi.2009.2038567 fatcat:bb2r67vnmnaztnrwikiisqx7qa

NDVI Short-Term Forecasting Using Recurrent Neural Networks

Arthur Stepchenko, Jurij Chizhov
2015 Environment Technology Resources Proceedings of the International Scientific and Practical Conference  
In this paper predictions of the Normalized Difference Vegetation Index (NDVI) data recorded by satellites over Ventspils Municipality in Courland, Latvia are discussed.  ...  In quasi-Newton methods, the idea is to use matrices which approximate the Hessian matrix and/or its inverse, instead of exact computing of the Hessian matrix (as in Newton-type methods).  ...  To achieve optimal weights of ERNN, BFGS quasi-Newton backpropogation algorithm provided by the MATLAB neural network toolbox is used to train the network.  ... 
doi:10.17770/etr2015vol3.167 fatcat:ha3lizdnkrbyvbdgjmg3wdnhoi

A Survey of Optimization Methods from a Machine Learning Perspective [article]

Shiliang Sun, Zehui Cao, Han Zhu, Jing Zhao
2019 arXiv   pre-print
machine learning research.  ...  Optimization, as an important part of machine learning, has attracted much attention of researchers.  ...  For example, Q-learning [184] and SARSA [185] are two popular methods which use temporal difference algorithms.  ... 
arXiv:1906.06821v2 fatcat:rcaas4ccpbdffhuvzcg2oryxr4

A Sparsification Approach for Temporal Graphical Model Decomposition

Ning Ruan, Ruoming Jin, Victor E. Lee, Kun Huang
2009 2009 Ninth IEEE International Conference on Data Mining  
We introduce an iterative optimization approach utilizing the Quasi-Newton method and generalized ridge regression to minimize the objective function and to produce a clustered temporal graphical model  ...  We also present a novel optimization procedure utilizing a graph theoretical tool based on the maximum weight independent set problem to speed up the Quasi-Newton method for a large number of variables  ...  The main issue is that the Quasi-Newton method is very costly for a large number of variables.  ... 
doi:10.1109/icdm.2009.67 pmid:23616730 pmcid:PMC3632353 dblp:conf/icdm/RuanJLH09 fatcat:slde5rymdnf6lbr62pq74mbzte

Solving Partial Differential Equations with Bernstein Neural Networks [chapter]

Sina Razvarz, Raheleh Jafari, Alexander Gegov
2018 Advances in Intelligent Systems and Computing  
This concept is laid down so as to produce a prevalent approximation on the basis of the learning method which is at par with quasi-Newton rule.  ...  For obtaining the superior estimated solution of PDEs, the adjustable parameters at par with the Bernstein neural network are adjusted in suitable manner by implementing quasi-Newton learning algorithm  ...  Numerical examples as well as comparison with solution obtained by the employing other numerical methodologies open up that the use of neural networks based on quasi-Newton learning rule provides solutions  ... 
doi:10.1007/978-3-319-97982-3_5 fatcat:3lnvpo6oabeupaoifyhrzuwxuy
« Previous Showing results 1 — 15 out of 6,351 results