Lower Bounds for Finding Stationary Points I
[article]
Yair Carmon, John C. Duchi, Oliver Hinder, Aaron Sidford
2019
arXiv
pre-print
We prove lower bounds on the complexity of finding ϵ-stationary points (points x such that ∇ f(x)<ϵ) of smooth, high-dimensional, and potentially non-convex functions f. We consider oracle-based complexity measures, where an algorithm is given access to the value and all derivatives of f at a query point x. We show that for any (potentially randomized) algorithm A, there exists a function f with Lipschitz pth order derivatives such that A requires at least ϵ^-(p+1)/p queries to find an
more »
... ary point. Our lower bounds are sharp to within constants, and they show that gradient descent, cubic-regularized Newton's method, and generalized pth order regularization are worst-case optimal within their natural function classes.
arXiv:1710.11606v3
fatcat:g5kkbx4tvzgupfvoihphl6iatm