Lower Bounds for Finding Stationary Points II: First-Order Methods [article]

Yair Carmon, John C. Duchi, Oliver Hinder, Aaron Sidford
2017 arXiv   pre-print
We establish lower bounds on the complexity of finding ϵ-stationary points of smooth, non-convex high-dimensional functions using first-order methods. We prove that deterministic first-order methods, even applied to arbitrarily smooth functions, cannot achieve convergence rates in ϵ better than ϵ^-8/5, which is within ϵ^-1/151/ϵ of the best known rate for such methods. Moreover, for functions with Lipschitz first and second derivatives, we prove no deterministic first-order method can achieve
more » ... nvergence rates better than ϵ^-12/7, while ϵ^-2 is a lower bound for functions with only Lipschitz gradient. For convex functions with Lipschitz gradient, accelerated gradient descent achieves the rate ϵ^-11/ϵ, showing that finding stationary points is easier given convexity.
arXiv:1711.00841v1 fatcat:pih7vinkybcxtggmbmzqwibxle