A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Adaptive Gradient Method with Resilience and Momentum
[article]
2020
arXiv
pre-print
Preserved Fulltext
In this paper, we proposed an Adaptive Gradient Method with Resilience and Momentum (AdaRem), motivated by the observation that the oscillations of network parameters slow the training, and give a theoretical ...
Although they show a large improvement in convergence speed, most adaptive learning rate methods suffer from compromised generalization compared with SGD. ...
To address this issue, we propose Adaptive Gradient Methods with REsilience and Momentum (AdaRem), a new adaptive optimization method that reduces useless oscillations by introducing damping. ...
arXiv:2010.11041v1
fatcat:cipglt4ckncrvgrpad4msl5jcm
A Comparative Study Of Backpropagation Algorithms In Financial Prediction
2011
International Journal of Computer Science Engineering and Applications
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
The accuracy of backpropagation neural networks trained with different heuristic and numerical algorithms is measured for comparison purpose. ...
Stock market price index prediction is a challenging task for investors and scholars. ...
The first category includes the gradient descent with adaptive learning rate, gradient descent with momentum, gradient descent with momentum and adaptive learning rate, and the resilient algorithm. ...
doi:10.5121/ijcsea.2011.1402
fatcat:x7igqvnmffb3xmsqdwmno4daum
Effect of training algorithms on neural networks aided pavement diagnosis
2010
International Journal of Engineering, Science and Technology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
In this paper, the effect of training algorithms on the NN aided inversion process is analyzed and discussed. ...
Efficient NN learning algorithms have been developed and proposed to determine the weights of the network, according to the data of the computational task to be performed. ...
Gradient Descent with Momentum and Adaptive Learning Rate Backpropagation (GDX) The GDX training algorithm combines adaptive learning rate with momentum training. ...
doi:10.4314/ijest.v2i2.59147
fatcat:7gi5mtre2jhc5ojjhqatfgx4au
Modified Gradient Search for Level Set Based Image Segmentation
2013
IEEE Transactions on Image Processing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
In this paper, we instead propose using two modified gradient descent methods, one using a momentum term and one based on resilient propagation. ...
The proposed methods are very simple modifications of the basic method, and are directly compatible with any type of level set implementation. ...
In this paper, we instead propose using two modified gradient descent methods, one using a momentum term and one based on resilient propagation. ...
doi:10.1109/tip.2012.2220148
pmid:23014748
fatcat:md7chiiqo5evbbfnpsssu7vj6e
Accelerating fuzzy clustering
2009
Information Sciences
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
This "gradient" may then be modified in the same way as a gradient is modified in error backpropagation in order to enhance the training. ...
Even though these modifications are, in principle, directly applicable, carefully checking and bounding the update steps can improve the performance and can make the procedure more robust. ...
Momentum Term The momentum term method [29] consists in adding a fraction of the weight change of the previous step to a normal gradient descent step. ...
doi:10.1016/j.ins.2008.09.017
fatcat:ibxyakyl7jhkvmy6ml3r6rvvde
An Artificial Neural Network Based Model For Predicting H2 Production Rates In A Sucrose-Based Bioreactor System
2008
Zenodo
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The efficient H2 production and system control can be provided by predictive control method combined with the robust BP based ANN modeling tool. ...
In this study, a predictive controller was proposed for management and operation of large scale H2-fermenting systems. The relevant control strategies can be activated by this method. ...
conjugate gradient 'traincgf'
0.096
0.901
27
Gradient descent with momentum and adaptive learning rate 'traingdx'
0.125
0.899
100
Levenberg-Marquardt 'trainlm'
0.014
0.897
10
Scaled conjugate ...
doi:10.5281/zenodo.1061137
fatcat:jlm63ypcq5b3ta3rrcltensyui
Distributed Momentum for Byzantine-resilient Learning
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We first prove that computing momentum at the workers reduces the variance-norm ratio of the gradient estimation at the server, strengthening Byzantine resilient aggregation rules. ...
Momentum is a variant of gradient descent that has been proposed for its benefits on convergence. In a distributed setting, momentum can be implemented either at the server or the worker side. ...
Any quantitative answer to this question will enable the use of our method in fully decentralised Byzantine resilient gradient descent.
A. ...
arXiv:2003.00010v2
fatcat:ykr3ay2jinbd3co3zfpd4lfefe
Robust Federated Recommendation System
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We then propose a robust learning strategy where instead of using model parameters, the central server computes and utilizes the gradients to filter out Byzantine clients. ...
Theoretically, we justify our robust learning strategy by our proposed definition of Byzantine resilience. ...
Since condition 1 and 2 of SGD with momentum-Byzantine resilience all hold, S-RFRS is SGD with momentum-Byzantine resilient. ...
arXiv:2006.08259v1
fatcat:boav3q2s5zgv5o3u5v5sxya6ti
Performance Evaluation of Training Algorithms in Backpropagation Neural Network Approach to Blast-Induced Ground Vibration Prediction
2020
Ghana Mining Journal
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
, Gradient Descent, and Gradient Descent with Momentum and Adaptive Learning Rate. ...
Gradient with Powell/Beale Restarts, Fletcher-Powell Conjugate Gradient, Polak-Ribiére Conjugate Gradient, One Step Secant, Gradient Descent with Adaptive Learning Rate, Gradient Descent with Momentum ...
authors would like to thank the Ghana National Petroleum Corporation (GNPC) for providing funding to support this work through the GNPC Professorial Chair in Mining Engineering at the University of Mines and ...
doi:10.4314/gm.v20i1.3
fatcat:gtw2jtqc2bg5hmqkfuy5okrgt4
A comparative study of breast cancer diagnosis based on neural network ensemble via improved training algorithms
2015
2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
with adaptive learning rate, BP with adaptive learning rate and momentum, Polak-Ribikre conjugate gradient algorithm (CGA), Fletcher-Reeves CGA, Powell-Beale CGA, scaled CGA, resilient BP (RBP), onestep ...
secant and quasi-Newton methods. ...
To overcome these limitations, BP with momentum, BP with adaptive learning rate, BP with adaptive learning rate and momentum, four kinds of conjugate gradient algorithms (CGAs), including Polak-Ribikre ...
doi:10.1109/embc.2015.7318982
pmid:26736882
dblp:conf/embc/AzamiE15a
fatcat:twqofvyanfbuxfcjuitch2tnpa
An Adaptive Stochastic Nesterov Accelerated Quasi Newton Method for Training RNNs
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The proposed method aSNAQ is an accelerated method that uses the Nesterov's gradient term along with second order curvature information. ...
This paper proposes a novel adaptive stochastic Nesterov accelerated quasiNewton (aSNAQ) method for training RNNs. ...
The proposed methodadaptive Stochastic Nesterov Accelerated Quasi-Newton (aSNAQ) incorporates Nesterov's accelerated gradient term and a simple adaptively tuned momentum term. ...
arXiv:1909.03620v1
fatcat:sosfmq27q5hmxidgtomsqmmjke
Optimization of CNN through Novel Training Strategy for Visual Classification Problems
2018
Entropy
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
descent with momentum (GDM). ...
For comparison, we present and analyze four different training algorithms for CNN along with MRPROP, i.e., resilient backpropagation (RPROP), Levenberg-Marquardt (LM), conjugate gradient (CG), and gradient ...
GDM: gradient descent with momentum; RPROP: resilient backpropagation; CG: conjugate gradient; LM: Levenberg-Marquardt. ...
doi:10.3390/e20040290
pmid:33265381
fatcat:ykahidspendbfc7a5cmt7z3kxi
A Study of Artificial Neural Network Training Algorithms for Classification of Cardiotocography Signals
2017
Bitlis Eren University Journal of Science and Technology
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Training algorithms of neural network were categorized in five group as Gradient Descent, Resilient Backpropagation, Conjugate Gradient, Quasi-Newton, and Levenberg-Marquardt. ...
In addition, the best classification performances were obtained with Levenberg-Marquardt backpropagation (LM) and Resilient Backpropagation (RP) algorithms. ...
descent with momentum and adaptive
learning rate backpropagation
2
RP
Resilient Backpropagation
3
CGF
Conjugate gradient backpropagation with
Fletcher-Reeves restarts
3
CGP
Conjugate gradient ...
doi:10.17678/beuscitech.338085
fatcat:yaymo652lrehbangkerbpinvja
The most accurate ANN learning algorithm for FEM prediction of mechanical performance of alloy A356
2012
Kovové materiály
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
Different primary and secondary dendrite arm spacings were used as inputs, and yield stress, UTS and elongation percentage were used as outputs in the training and test modules of the neural network. ...
After the preparation of the training set, the neural network was trained using different training algorithms, hidden layers and neuron numbers in hidden layers. ...
-Gradient descent with momentum and adaptive learning rule back propagation: is a network training function that updates weight and bias values accord- ing to gradient descent momentum and an adaptive ...
doi:10.4149/km_2012_1_25
fatcat:oqtpyg54jvcyjccyocl7tp3t4m
RES-HD: Resilient Intelligent Fault Diagnosis Against Adversarial Attacks Using Hyper-Dimensional Computing
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Our experiments show that HDC leads to a more resilient and lightweight learning solution than the state-of-the-art deep learning methods. ...
The change in the classification accuracy is measured as the difference before and after the attacks. This change measures the resiliency of a learning method. ...
Momentum Iterative Method (MIM) Momentum Iterative Method (MIM) solves underfitting and overfitting problems in FGSM and BIM respectively by integrating momentum into the BIM [31] . ...
arXiv:2203.08148v1
fatcat:tcvner27jvg65ou5hzlxvmu5te
« Previous
Showing results 1 — 15 out of 4,491 results