Cramer-Rao bounds and Monte Carlo calculation of the Fisher information matrix in difficult problems

J.C. Spall
2004 Proceedings of the 2004 American Control Conference   unpublished
and the JHU/APL IRAD Program. This paper extends results presented in preliminary form at the 2003 ACC to provide more theoretical justification for the Monte Carlo method (Sect. 4), to introduce the use of antithetic random numbers (Sect. 5), and to carry out a new numerical study (Sect. 6). A more complete version of this paper is available upon request. Abstract: The Fisher information matrix summarizes the amount of information in the data relative to the quantities of interest. There are
more » ... terest. There are many applications of the information matrix in modeling, systems analysis, and estimation, including confidence region calculation, input design, prediction bounds, and "noninformative" priors for Bayesian analysis. This paper reviews some basic principles associated with the information matrix, presents a resampling-based method for computing the information matrix together with some new theory related to efficient implementation, and presents some numerical results. The resampling-based method relies on an efficient technique for estimating the Hessian matrix, introduced as part of the adaptive ("secondorder") form of the simultaneous perturbation stochastic approximation (SPSA) optimization algorithm.
doi:10.23919/acc.2004.1384392 fatcat:ubrqcev6ovcxjidcy4cstd2f4u