A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf
.
An Adaptive Remote Stochastic Gradient Method for Training Neural Networks
[article]
2020
arXiv
pre-print
We present the remote stochastic gradient (RSG) method, which computes the gradients at configurable remote observation points, in order to improve the convergence rate and suppress gradient noise at the same time for different curvatures. RSG is further combined with adaptive methods to construct ARSG for acceleration. The method is efficient in computation and memory, and is straightforward to implement. We analyze the convergence properties by modeling the training process as a dynamic
arXiv:1905.01422v8
fatcat:7l5cewu2dne4rg3xujhqf3ad7q