A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf
.
Uniform Learning in a Deep Neural Network via "Oddball" Stochastic Gradient Descent
[article]
2015
arXiv
pre-print
When training deep neural networks, it is typically assumed that the training examples are uniformly difficult to learn. Or, to restate, it is assumed that the training error will be uniformly distributed across the training examples. Based on these assumptions, each training example is used an equal number of times. However, this assumption may not be valid in many cases. "Oddball SGD" (novelty-driven stochastic gradient descent) was recently introduced to drive training probabilistically
arXiv:1510.02442v1
fatcat:6szvh2jpirbldpnsxew4ybcuda