4,491 Hits in 2.9 sec

Adaptive Gradient Method with Resilience and Momentum [article]

Jie Liu, Chen Lin, Chuming Li, Lu Sheng, Ming Sun, Junjie Yan, Wanli Ouyang
2020 arXiv   pre-print
In this paper, we proposed an Adaptive Gradient Method with Resilience and Momentum (AdaRem), motivated by the observation that the oscillations of network parameters slow the training, and give a theoretical  ...  Although they show a large improvement in convergence speed, most adaptive learning rate methods suffer from compromised generalization compared with SGD.  ...  To address this issue, we propose Adaptive Gradient Methods with REsilience and Momentum (AdaRem), a new adaptive optimization method that reduces useless oscillations by introducing damping.  ... 
arXiv:2010.11041v1 fatcat:cipglt4ckncrvgrpad4msl5jcm

A Comparative Study Of Backpropagation Algorithms In Financial Prediction

Salim Lahmiri
2011 International Journal of Computer Science Engineering and Applications  
The accuracy of backpropagation neural networks trained with different heuristic and numerical algorithms is measured for comparison purpose.  ...  Stock market price index prediction is a challenging task for investors and scholars.  ...  The first category includes the gradient descent with adaptive learning rate, gradient descent with momentum, gradient descent with momentum and adaptive learning rate, and the resilient algorithm.  ... 
doi:10.5121/ijcsea.2011.1402 fatcat:x7igqvnmffb3xmsqdwmno4daum

Effect of training algorithms on neural networks aided pavement diagnosis

K Gopalakrishnan
2010 International Journal of Engineering, Science and Technology  
In this paper, the effect of training algorithms on the NN aided inversion process is analyzed and discussed.  ...  Efficient NN learning algorithms have been developed and proposed to determine the weights of the network, according to the data of the computational task to be performed.  ...  Gradient Descent with Momentum and Adaptive Learning Rate Backpropagation (GDX) The GDX training algorithm combines adaptive learning rate with momentum training.  ... 
doi:10.4314/ijest.v2i2.59147 fatcat:7gi5mtre2jhc5ojjhqatfgx4au

Modified Gradient Search for Level Set Based Image Segmentation

T. Andersson, G. Lathen, R. Lenz, M. Borga
2013 IEEE Transactions on Image Processing  
In this paper, we instead propose using two modified gradient descent methods, one using a momentum term and one based on resilient propagation.  ...  The proposed methods are very simple modifications of the basic method, and are directly compatible with any type of level set implementation.  ...  In this paper, we instead propose using two modified gradient descent methods, one using a momentum term and one based on resilient propagation.  ... 
doi:10.1109/tip.2012.2220148 pmid:23014748 fatcat:md7chiiqo5evbbfnpsssu7vj6e

Accelerating fuzzy clustering

Christian Borgelt
2009 Information Sciences  
This "gradient" may then be modified in the same way as a gradient is modified in error backpropagation in order to enhance the training.  ...  Even though these modifications are, in principle, directly applicable, carefully checking and bounding the update steps can improve the performance and can make the procedure more robust.  ...  Momentum Term The momentum term method [29] consists in adding a fraction of the weight change of the previous step to a normal gradient descent step.  ... 
doi:10.1016/j.ins.2008.09.017 fatcat:ibxyakyl7jhkvmy6ml3r6rvvde

An Artificial Neural Network Based Model For Predicting H2 Production Rates In A Sucrose-Based Bioreactor System

Nikhil, Bestamin Özkaya, Ari Visa, Chiu-Yue Lin, Jaakko A. Puhakka, Olli Yli-Harja
2008 Zenodo  
The efficient H2 production and system control can be provided by predictive control method combined with the robust BP based ANN modeling tool.  ...  In this study, a predictive controller was proposed for management and operation of large scale H2-fermenting systems. The relevant control strategies can be activated by this method.  ...  conjugate gradient 'traincgf' 0.096 0.901 27 Gradient descent with momentum and adaptive learning rate 'traingdx' 0.125 0.899 100 Levenberg-Marquardt 'trainlm' 0.014 0.897 10 Scaled conjugate  ... 
doi:10.5281/zenodo.1061137 fatcat:jlm63ypcq5b3ta3rrcltensyui

Distributed Momentum for Byzantine-resilient Learning [article]

El-Mahdi El-Mhamdi, Rachid Guerraoui, Sébastien Rouault
2020 arXiv   pre-print
We first prove that computing momentum at the workers reduces the variance-norm ratio of the gradient estimation at the server, strengthening Byzantine resilient aggregation rules.  ...  Momentum is a variant of gradient descent that has been proposed for its benefits on convergence. In a distributed setting, momentum can be implemented either at the server or the worker side.  ...  Any quantitative answer to this question will enable the use of our method in fully decentralised Byzantine resilient gradient descent. A.  ... 
arXiv:2003.00010v2 fatcat:ykr3ay2jinbd3co3zfpd4lfefe

Robust Federated Recommendation System [article]

Chen Chen, Jingfeng Zhang, Anthony K. H. Tung, Mohan Kankanhalli, Gang Chen
2020 arXiv   pre-print
We then propose a robust learning strategy where instead of using model parameters, the central server computes and utilizes the gradients to filter out Byzantine clients.  ...  Theoretically, we justify our robust learning strategy by our proposed definition of Byzantine resilience.  ...  Since condition 1 and 2 of SGD with momentum-Byzantine resilience all hold, S-RFRS is SGD with momentum-Byzantine resilient.  ... 
arXiv:2006.08259v1 fatcat:boav3q2s5zgv5o3u5v5sxya6ti

Performance Evaluation of Training Algorithms in Backpropagation Neural Network Approach to Blast-Induced Ground Vibration Prediction

C. K. Arthur, V. A. Temeng, Y. Y. Ziggah
2020 Ghana Mining Journal  
, Gradient Descent, and Gradient Descent with Momentum and Adaptive Learning Rate.  ...  Gradient with Powell/Beale Restarts, Fletcher-Powell Conjugate Gradient, Polak-Ribiére Conjugate Gradient, One Step Secant, Gradient Descent with Adaptive Learning Rate, Gradient Descent with Momentum  ...  authors would like to thank the Ghana National Petroleum Corporation (GNPC) for providing funding to support this work through the GNPC Professorial Chair in Mining Engineering at the University of Mines and  ... 
doi:10.4314/gm.v20i1.3 fatcat:gtw2jtqc2bg5hmqkfuy5okrgt4

A comparative study of breast cancer diagnosis based on neural network ensemble via improved training algorithms

Hamed Azami, Javier Escudero
2015 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)  
with adaptive learning rate, BP with adaptive learning rate and momentum, Polak-Ribikre conjugate gradient algorithm (CGA), Fletcher-Reeves CGA, Powell-Beale CGA, scaled CGA, resilient BP (RBP), onestep  ...  secant and quasi-Newton methods.  ...  To overcome these limitations, BP with momentum, BP with adaptive learning rate, BP with adaptive learning rate and momentum, four kinds of conjugate gradient algorithms (CGAs), including Polak-Ribikre  ... 
doi:10.1109/embc.2015.7318982 pmid:26736882 dblp:conf/embc/AzamiE15a fatcat:twqofvyanfbuxfcjuitch2tnpa

An Adaptive Stochastic Nesterov Accelerated Quasi Newton Method for Training RNNs [article]

S. Indrapriyadarsini, Shahrzad Mahboubi, Hiroshi Ninomiya, Hideki Asai
2019 arXiv   pre-print
The proposed method aSNAQ is an accelerated method that uses the Nesterov's gradient term along with second order curvature information.  ...  This paper proposes a novel adaptive stochastic Nesterov accelerated quasiNewton (aSNAQ) method for training RNNs.  ...  The proposed methodadaptive Stochastic Nesterov Accelerated Quasi-Newton (aSNAQ) incorporates Nesterov's accelerated gradient term and a simple adaptively tuned momentum term.  ... 
arXiv:1909.03620v1 fatcat:sosfmq27q5hmxidgtomsqmmjke

Optimization of CNN through Novel Training Strategy for Visual Classification Problems

Sadaqat Rehman, Shanshan Tu, Obaid Rehman, Yongfeng Huang, Chathura Magurawalage, Chin-Chen Chang
2018 Entropy  
descent with momentum (GDM).  ...  For comparison, we present and analyze four different training algorithms for CNN along with MRPROP, i.e., resilient backpropagation (RPROP), Levenberg-Marquardt (LM), conjugate gradient (CG), and gradient  ...  GDM: gradient descent with momentum; RPROP: resilient backpropagation; CG: conjugate gradient; LM: Levenberg-Marquardt.  ... 
doi:10.3390/e20040290 pmid:33265381 fatcat:ykahidspendbfc7a5cmt7z3kxi

A Study of Artificial Neural Network Training Algorithms for Classification of Cardiotocography Signals

2017 Bitlis Eren University Journal of Science and Technology  
Training algorithms of neural network were categorized in five group as Gradient Descent, Resilient Backpropagation, Conjugate Gradient, Quasi-Newton, and Levenberg-Marquardt.  ...  In addition, the best classification performances were obtained with Levenberg-Marquardt backpropagation (LM) and Resilient Backpropagation (RP) algorithms.  ...  descent with momentum and adaptive learning rate backpropagation 2 RP Resilient Backpropagation 3 CGF Conjugate gradient backpropagation with Fletcher-Reeves restarts 3 CGP Conjugate gradient  ... 
doi:10.17678/beuscitech.338085 fatcat:yaymo652lrehbangkerbpinvja

The most accurate ANN learning algorithm for FEM prediction of mechanical performance of alloy A356

2012 Kovové materiály  
Different primary and secondary dendrite arm spacings were used as inputs, and yield stress, UTS and elongation percentage were used as outputs in the training and test modules of the neural network.  ...  After the preparation of the training set, the neural network was trained using different training algorithms, hidden layers and neuron numbers in hidden layers.  ...  -Gradient descent with momentum and adaptive learning rule back propagation: is a network training function that updates weight and bias values accord- ing to gradient descent momentum and an adaptive  ... 
doi:10.4149/km_2012_1_25 fatcat:oqtpyg54jvcyjccyocl7tp3t4m

RES-HD: Resilient Intelligent Fault Diagnosis Against Adversarial Attacks Using Hyper-Dimensional Computing [article]

Onat Gungor, Tajana Rosing, Baris Aksanli
2022 arXiv   pre-print
Our experiments show that HDC leads to a more resilient and lightweight learning solution than the state-of-the-art deep learning methods.  ...  The change in the classification accuracy is measured as the difference before and after the attacks. This change measures the resiliency of a learning method.  ...  Momentum Iterative Method (MIM) Momentum Iterative Method (MIM) solves underfitting and overfitting problems in FGSM and BIM respectively by integrating momentum into the BIM [31] .  ... 
arXiv:2203.08148v1 fatcat:tcvner27jvg65ou5hzlxvmu5te
« Previous Showing results 1 — 15 out of 4,491 results