ePCA: High dimensional exponential family PCA

Lydia T. Liu, Edgar Dobriban, Amit Singer

Research output: Contribution to journalArticle

7 Scopus citations

Abstract

Many applications involve large datasets with entries from exponential family distributions. Our main motivating application is photon-limited imaging, where we observe images with Poisson distributed pixels. We focus on X-ray Free Electron Lasers (XFEL), a quickly developing technology whose goal is to reconstruct molecular structure. In XFEL, estimating the principal components of the noiseless distribution is needed for denoising and for structure determination. However, the standard method, Principal Component Analysis (PCA), can be inefficient in non-Gaussian noise. Motivated by this application, we develop ePCA (exponential family PCA), a new methodology for PCA on exponential families. ePCA is a fast method that can be used very generally for dimension reduction and denoising of large data matrices with exponential family entries. We conduct a substantive XFEL data analysis using ePCA. We show that ePCA estimates the PCs of the distribution of images more accurately than PCA and alternatives. Importantly, it also leads to better denoising. We also provide theoretical justification for our estimator, including the convergence rate and the Marchenko–Pastur law in high dimensions. An open-source implementation is available.

Original languageEnglish (US)
Pages (from-to)2121-2150
Number of pages30
JournalAnnals of Applied Statistics
Volume12
Issue number4
DOIs
StatePublished - Dec 2018

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Modeling and Simulation
  • Statistics, Probability and Uncertainty

Keywords

  • Denoising
  • Random matrix theory
  • Shrinkage
  • Xfel imaging

Fingerprint Dive into the research topics of 'ePCA: High dimensional exponential family PCA'. Together they form a unique fingerprint.

  • Cite this