Hidden Markov model multiarm bandits: a methodology for beam scheduling in multitarget tracking

V. Krishnamurthy, R.J. Evans
2001 IEEE Transactions on Signal Processing  
In this paper, we derive optimal and suboptimal beam scheduling algorithms for electronically scanned array tracking systems. We formulate the scheduling problem as a multiarm bandit problem involving hidden Markov models (HMMs). A finite-dimensional optimal solution to this multiarm bandit problem is presented. The key to solving any multiarm bandit problem is to compute the Gittins index. We present a finite-dimensional algorithm that computes the Gittins index. Suboptimal algorithms for
more » ... ting the Gittins index are also presented. Numerical examples are presented to illustrate the algorithms. Index Terms-Dynamic programming, hidden Markov models, optimal beam steering, scheduling, sequential decision procedures.
doi:10.1109/78.969499 fatcat:6fzkiqvuibfnjjze7mfofs2ffq