In this paper, we derive optimal and suboptimal beam scheduling algorithms for electronically scanned array tracking systems. We formulate the scheduling problem as a multiarm bandit problem involving hidden Markov models (HMMs). A finite-dimensional optimal solution to this multiarm bandit problem is presented. The key to solving any multiarm bandit problem is to compute the Gittins index. We present a finite-dimensional algorithm that computes the Gittins index. Suboptimal algorithms for

more »

... ting the Gittins index are also presented. Numerical examples are presented to illustrate the algorithms. Index Terms-Dynamic programming, hidden Markov models, optimal beam steering, scheduling, sequential decision procedures.