Large system spectral analysis of covariance matrix estimation

Eigendecomposition of estimated covariance matrices is a basic signal processing technique arising in a number of applications, including direction-of-arrival estimation, power allocation in multiple-input/ multiple-output (MIMO) transmission systems, and adaptive multiuser detection. This paper uses the theory of non-crossing partitions to develop explicit asymptotic expressions for the moments of the eigenvalues of estimated covariance matrices, in the large system asymptote as the vector dimension and the dimension of signal space both increase without bound, while their ratio remains finite and nonzero. The asymptotic eigenvalue distribution is also obtained from these eigenvalue moments and the Stieltjes transform, and is extended to first-order approximation in the large sample-size limit. Numerical simulations are used to demonstrate that these asymptotic results provide good approximations for finite systems of moderate size.