TY - JOUR
T1 - Distributed Estimation for Principal Component Analysis
T2 - An Enlarged Eigenspace Analysis
AU - Chen, Xi
AU - Lee, Jason D.
AU - Li, He
AU - Yang, Yun
N1 - Funding Information:
Xi Chen is supported by NSF grant IIS-1845444. Jason D. Lee is supported by NSF grant CCF-2002272. Yun Yang is supported by NSF grant DMS-1810831.
Publisher Copyright:
© 2021 American Statistical Association.
PY - 2021
Y1 - 2021
N2 - The growing size of modern datasets brings many challenges to the existing statistical estimation approaches, which calls for new distributed methodologies. This article studies distributed estimation for a fundamental statistical machine learning problem, principal component analysis (PCA). Despite the massive literature on top eigenvector estimation, much less is presented for the top-L-dim (L > 1) eigenspace estimation, especially in a distributed manner. We propose a novel multi-round algorithm for constructing top-L-dim eigenspace for distributed data. Our algorithm takes advantage of shift-and-invert preconditioning and convex optimization. Our estimator is communication-efficient and achieves a fast convergence rate. In contrast to the existing divide-and-conquer algorithm, our approach has no restriction on the number of machines. Theoretically, the traditional Davis–Kahan theorem requires the explicit eigengap assumption to estimate the top-L-dim eigenspace. To abandon this eigengap assumption, we consider a new route in our analysis: instead of exactly identifying the top-L-dim eigenspace, we show that our estimator is able to cover the targeted top-L-dim population eigenspace. Our distributed algorithm can be applied to a wide range of statistical problems based on PCA, such as principal component regression and single index model. Finally, we provide simulation studies to demonstrate the performance of the proposed distributed estimator.
AB - The growing size of modern datasets brings many challenges to the existing statistical estimation approaches, which calls for new distributed methodologies. This article studies distributed estimation for a fundamental statistical machine learning problem, principal component analysis (PCA). Despite the massive literature on top eigenvector estimation, much less is presented for the top-L-dim (L > 1) eigenspace estimation, especially in a distributed manner. We propose a novel multi-round algorithm for constructing top-L-dim eigenspace for distributed data. Our algorithm takes advantage of shift-and-invert preconditioning and convex optimization. Our estimator is communication-efficient and achieves a fast convergence rate. In contrast to the existing divide-and-conquer algorithm, our approach has no restriction on the number of machines. Theoretically, the traditional Davis–Kahan theorem requires the explicit eigengap assumption to estimate the top-L-dim eigenspace. To abandon this eigengap assumption, we consider a new route in our analysis: instead of exactly identifying the top-L-dim eigenspace, we show that our estimator is able to cover the targeted top-L-dim population eigenspace. Our distributed algorithm can be applied to a wide range of statistical problems based on PCA, such as principal component regression and single index model. Finally, we provide simulation studies to demonstrate the performance of the proposed distributed estimator.
KW - Convergence analysis
KW - Distributed estimation
KW - Enlarged eigenspace
KW - Principal component analysis
KW - Shift-and-invert preconditioning
UR - http://www.scopus.com/inward/record.url?scp=85103748581&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85103748581&partnerID=8YFLogxK
U2 - 10.1080/01621459.2021.1886937
DO - 10.1080/01621459.2021.1886937
M3 - Article
AN - SCOPUS:85103748581
SN - 0162-1459
JO - Journal of the American Statistical Association
JF - Journal of the American Statistical Association
ER -