High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence

Pradeep Ravikumar, Martin J. Wainwright, Garvesh Raskutti, Bin Yu
2011 Electronic Journal of Statistics  
Given i.i.d. observations of a random vector X ∈ R p , we study the problem of estimating both its covariance matrix Σ * , and its inverse covariance or concentration matrix Θ * = (Σ * ) −1 . When X is multivariate Gaussian, the non-zero structure of Θ * is specified by the graph of an associated Gaussian Markov random field; and a popular estimator for such sparse Θ * is the ℓ 1 -regularized Gaussian MLE. This estimator is sensible even for for non-Gaussian X, since it corresponds to
more » ... sponds to minimizing an ℓ 1 -penalized log-determinant Bregman divergence. We analyze its performance under high-dimensional scaling, in which the number of nodes in the graph p, the number of edges s, and the maximum node degree d, are allowed to grow as a function of the sample size n. In addition to the parameters (p, s, d), our analysis identifies other key quantities that control rates: (a) the ℓ∞-operator norm of the true covariance matrix Σ * ; and (b) the ℓ∞ operator norm of the sub-matrix Γ * SS , where S indexes the graph edges, and Γ * = (Θ * ) −1 ⊗ (Θ * ) −1 ; and (c) a mutual incoherence or irrepresentability measure on the matrix Γ * and (d) the rate of decay 1/f (n, δ) on the probabilities {| Σ n ij − Σ * ij | > δ}, where Σ n is the sample covariance based on n samples. Our first result establishes consistency of our estimate Θ in the elementwise maximum-norm. This in turn allows us to derive convergence rates in Frobenius and spectral norms, with improvements upon existing results for graphs with maximum node degrees d = o( √ s). In our second result, we show that with probability converging to one, the estimate Θ correctly specifies the zero pattern of the concentration matrix Θ * . We illustrate our theoretical results via simulations for various graphs and problem parameters, showing good correspondences between the theoretical predictions and behavior in simulations.
doi:10.1214/11-ejs631 fatcat:cclcuirfrrb4tfub5m72smas2e