Capon algorithm mean-squared error threshold SNR prediction and probability of resolution

C.D. Richmond
2005 IEEE Transactions on Signal Processing  
Below a specific threshold signal-to-noise ratio (SNR), the mean-squared error (MSE) performance of signal parameter estimates derived from the Capon algorithm degrades swiftly. Prediction of this threshold SNR point is of practical significance for robust system design and analysis. The exact pairwise error probabilities for the Capon (and Bartlett) algorithm, derived herein, are given by simple finite sums involving no numerical integration, include finite sample effects, and hold for an
more » ... nd hold for an arbitrary colored data covariance. Via an adaptation of an interval error based method, these error probabilities, along with the local error MSE predictions of Vaidyanathan and Buckley, facilitate accurate prediction of the Capon threshold region MSE performance for an arbitrary number of well separated sources, circumventing the need for numerous Monte Carlo simulations. A large sample closed-form approximation for the Capon threshold SNR is provided for uniform linear arrays. A new, exact, two-point measure of the probability of resolution for the Capon algorithm, that includes the deleterious effects of signal model mismatch, is a serendipitous byproduct of this analysis that predicts the SNRs required for closely spaced sources to be mutually resolvable by the Capon algorithm. Last, a general strategy is provided for obtaining accurate MSE predictions that account for signal model mismatch. Index Terms-Adaptive, beamformer, Bartlett, Capon, complex Wishart, direction-of-arrival (DOA), DOA estimation, finite sample, mean-squared error (MSE), mismatch, probability of resolution, resolution, sensitivity, threshold, threshold signal-to-noise ratio (SNR), Wishart. 1053-587X/$20.00 © 2005 IEEE Christ D. Richmond (M'99-SM'05) received the B.S. degree in electrical engineering from the University of Maryland, College Park, the B.S. degree in mathematics from Bowie State University, Bowie, MD, and the M.S., E.E., and Ph.D. degrees in electrical engineering from the Massachusetts Institute of Technology (MIT), Cambridge. He is currently a member of the technical research staff at the MIT Lincoln Laboratory, Lexington. His research interests include detection and parameter estimation theory, statistical signal and array processing, radar/sonar signal processing, multivariate statistical analysis, multiuser detection, multi-input multi-output (MIMO) systems, and wireless communications, and random matrix theory. Dr. Richmond is the recipient of the Office
doi:10.1109/tsp.2005.850361 fatcat:zxaatc3werex7iojwkb6gewwhi