Embracing the Blessing of Dimensionality in Factor Models

Quefeng Li, Guang Cheng, Jianqing Fan, Yuyan Wang

Research output: Contribution to journalArticle

4 Scopus citations

Abstract

Factor modeling is an essential tool for exploring intrinsic dependence structures among high-dimensional random variables. Much progress has been made for estimating the covariance matrix from a high-dimensional factor model. However, the blessing of dimensionality has not yet been fully embraced in the literature: much of the available data are often ignored in constructing covariance matrix estimates. If our goal is to accurately estimate a covariance matrix of a set of targeted variables, shall we employ additional data, which are beyond the variables of interest, in the estimation? In this article, we provide sufficient conditions for an affirmative answer, and further quantify its gain in terms of Fisher information and convergence rate. In fact, even an oracle-like result (as if all the factors were known) can be achieved when a sufficiently large number of variables is used. The idea of using data as much as possible brings computational challenges. A divide-and-conquer algorithm is thus proposed to alleviate the computational burden, and also shown not to sacrifice any statistical accuracy in comparison with a pooled analysis. Simulation studies further confirm our advocacy for the use of full data, and demonstrate the effectiveness of the above algorithm. Our proposal is applied to a microarray data example that shows empirical benefits of using more data. Supplementary materials for this article are available online.

Original languageEnglish (US)
Pages (from-to)380-389
Number of pages10
JournalJournal of the American Statistical Association
Volume113
Issue number521
DOIs
StatePublished - Jan 2 2018

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Keywords

  • Asymptotic normality
  • Auxiliary data
  • Divide-and-conquer
  • Factor model
  • Fisher information
  • High-dimensionality

Fingerprint Dive into the research topics of 'Embracing the Blessing of Dimensionality in Factor Models'. Together they form a unique fingerprint.

  • Cite this