Detection and Remediation of Stagnation in the Nelder--Mead Algorithm Using a Sufficient Decrease Condition

C. T. Kelley
1999 SIAM Journal on Optimization  
The Nelder-Mead algorithm can stagnate and converge to a non-optimal point, even for very simple problems. In this note we propose a test for sufficient decrease which, if passed for the entire iteration, will guarantee convergence of the Nelder-Mead iteration to a stationary point if the objective function is smooth. Failure of this condition is an indicator of potential stagnation. As a remedy we propose a new step, which we call an oriented restart, which reinitializes the simplex to a
more » ... r one with orthogonal edges which contains an approximate steepest descent step from the current best point. We also give results that apply when the objective function is a lowamplitude perturbation of a smooth function. We illustrate our results with some numerical examples.
doi:10.1137/s1052623497315203 fatcat:yldulfbxh5ci3jriu4yi3rrwa4