A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf
.
The Implicit Regularization of Stochastic Gradient Flow for Least Squares
[article]
2020
arXiv
pre-print
We study the implicit regularization of mini-batch stochastic gradient descent, when applied to the fundamental problem of least squares regression. We leverage a continuous-time stochastic differential equation having the same moments as stochastic gradient descent, which we call stochastic gradient flow. We give a bound on the excess risk of stochastic gradient flow at time t, over ridge regression with tuning parameter λ = 1/t. The bound may be computed from explicit constants (e.g., the
arXiv:2003.07802v2
fatcat:req5g6wjlncsbexknakh7kbyii