On the best choice of a damping sequence in iterative optimization methods

L. N. Vaserstein
1988 Publicacions matemàtiques  
1. Iterative procedure xt = (1 -at)xt-1 + atyt = xt-1 + at(ytxt-1), f([xt-1,yt])f(X) <-e(f(xt-1) -f(X)) Some iterative methods of mathematical programming use a damping sequence {cat} such that 0 _< at < 1 for all t, at -0 as too, and at = oo . For example, at = 1l(t + 1) in Brown's method for solving matrix games. In this paper, for a model class of iterative methods, the convergente rate for any damping sequence {cet } depending only on time t is computed . This computation is used to find
more » ... is used to find the best damping sequence . Let L be a real affine space (so L with an origin fixed is the same as a real vector space) . For any points x and y in L, let [x, y] denote the closed interval with the ends x and y. For any real-valued function f on a subset X of L, let f(X) denote its infimum on X . On a non-empty subset X of L, we consider an iterative procedure of the form where 0 < at < 1, 1 < t < T + 1, and [yt , xt_1] C X. Here the total number T of iterations is either finite or infinite (T = oo); in the second case t runs over all natural numbers . The objective of the procedure, starting at a point xo of X, is to minimize a convex bounded from below function f on X. (We call f convex on X, if its restriction on every interval contained in X is convex) . To reach this objective, at each step t, one tends to choose yt in X, so that f decreases when one starts to move from xt_1 to yt . The choice of yt depends, in general, on f, x t_1, and t. We abstract ourselves from any concrete rule of choosing yt , and just assume that the choice was good enough . Namely, we fix a number 0 in the interval 0 <_ 0 < 1 and consider the class of iterative methods such that
doi:10.5565/publmat_32288_11 fatcat:da42wvlhyrdyfml5vhdkuskjhe