Mixing times with applications to perturbed Markov chains

Jeffrey J. Hunter
2006 Linear Algebra and its Applications  
A measure of the "mixing time" or "time to stationarity" in a finite irreducible discrete time Markov chain is considered. The statistic η π i i j j m j m = = ∑ 1 , where {π j } is the stationary distribution and m ij is the mean first passage time from state i to state j of the Markov chain, is shown to be independent of the state i that the chain starts in (so that η i = η for all i), is minimal in the case of a periodic chain, yet can be arbitrarily large in a variety of situations. An
more » ... ation concerning the affect perturbations of transition probabilities have on the stationary distributions of Markov chains leads to a new bound, involving η, for the 1-norm of the difference between the stationary probability vectors of the original and the perturbed chain. When η is large the stationary distribution of the Markov chain is very sensitive to perturbations of the transition probabilities. Mixing Times Let P = [p ij ] be the transition matrix of a finite irreducible, discrete time Markov chain {X n }, (n ≥ 0), with state space S = {1, 2,..., m}. Let {π j }, (1 ≤ j ≤ m), be the stationary distribution of the chain and π π π π ′ = (π 1 , π 2 ,... ,π m ) its stationary probability vector. As noted earlier, for all regular Markov chains, for all j ∈S, lim n→∞ P[X n = j] = π j . For all irreducible chains (including periodic chains), if for some k ≥ 0, P[X k = j] = π j for all j ∈S, then P[X n = j] = π j for all n ≥ k and all j ∈S. Let T ij be the first passage time random variable from state i to state j, i.e. T ij = min{ n ≥ 1 such that X n = j given that X 0 = i}. (In some writings this r.v. is given a superscript + to denote that the minimum is taken over n ≥ 1 rather than n ≥ 0. This distinction is only of interest when i = j. In this case, we consider T ii to be the "first return to state i" as opposed to the "first hitting time of state i"). Let M = [m ij ] be the matrix of the mean first passage times from state i to state j, i.e. m ij = E[T ij | X 0 = i] for all i, j ∈S. Once the Markov chain achieves stationarity, at say step n, the distribution of X n can be assumed to be the stationary distribution, i.e. P[X n = j] = π j for each j ∈S. If that is the case then it easy to show that, for all k ≥ n, P[X k = j] = π j for each j ∈S. 36 R.L.I.M.S. Vol. 4, May 2003 Definition 2.1: (T, the time to "mixing" in a Markov chain) Let Y be a random variable whose probability distribution is the stationary distribution {π j }. We shall say that the Markov chain {X n }, "reaches stationarity", or achieves "mixing", at time T = k, when X k = Y for the smallest such k ≥ 1. ❑ Thus, we first sample from the stationary distribution {π j } to determine a value of the random variable Y, say Y = j. We then observe the Markov chain, starting at a given state i and achieve "mixing" at time T = n when X n = j for the first such n ≥ 1. i.e., conditional upon Y = j, T = T ij , the first passage time from state i to state j.
doi:10.1016/j.laa.2006.02.008 fatcat:s2tgcafr6ncbpmkrt3dllr4o3a