Lower Bounds for Finding Stationary Points I [article]

Yair Carmon, John C. Duchi, Oliver Hinder, Aaron Sidford
2019 arXiv   pre-print
We prove lower bounds on the complexity of finding ϵ-stationary points (points x such that ∇ f(x)<ϵ) of smooth, high-dimensional, and potentially non-convex functions f. We consider oracle-based complexity measures, where an algorithm is given access to the value and all derivatives of f at a query point x. We show that for any (potentially randomized) algorithm A, there exists a function f with Lipschitz pth order derivatives such that A requires at least ϵ^-(p+1)/p queries to find an
more » ... ary point. Our lower bounds are sharp to within constants, and they show that gradient descent, cubic-regularized Newton's method, and generalized pth order regularization are worst-case optimal within their natural function classes.
arXiv:1710.11606v3 fatcat:g5kkbx4tvzgupfvoihphl6iatm