Bayesian group factor analysis with structured sparsity

Shiwen Zhao, Chuan Gao, Sayan Mukherjee, Barbara Engelhardt Martin

Research output: Contribution to journalArticlepeer-review

38 Scopus citations


Latent factor models are the canonical statistical tool for exploratory analyses of lowdimensional linear structure for a matrix of p features across n samples. We develop a structured Bayesian group factor analysis model that extends the factor model to multiple coupled observation matrices; in the case of two observations, this reduces to a Bayesian model of canonical correlation analysis. Here, we carefully de-ne a structured Bayesian prior that encourages both element-wise and column-wise shrinkage and leads to desirable behavior on high-dimensional data. In particular, our model puts a structured prior on the joint factor loading matrix, regularizing at three levels, which enables element-wise sparsity and unsupervised recovery of latent factors corresponding to structured variance across arbitrary subsets of the observations. In addition, our structured prior allows for both dense and sparse latent factors so that covariation among either all features or only a subset of features can be recovered. We use fast parameter-expanded expectation-maximization for parameter estimation in this model. We validate our method on simulated data with substantial structure. We show results of our method applied to three high-dimensional data sets, comparing results against a number of state-of-The-Art approaches. These results illustrate useful properties of our model, including i) recovering sparse signal in the presence of dense effects; ii) the ability to scale naturally to large numbers of observations; iii) exible observation-and factor-specific regularization to recover factors with a wide variety of sparsity levels and percentage of variance explained; and iv) tractable inference that scales to modern genomic and text data sizes.

Original languageEnglish (US)
Pages (from-to)1-47
Number of pages47
JournalJournal of Machine Learning Research
StatePublished - Apr 1 2016

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence
  • Control and Systems Engineering
  • Statistics and Probability


  • Bayesian Structured Sparsity
  • Canonical Correlation Analysis
  • Mixture Models
  • Parameter Expansion
  • Sparse And Low-Rank Matrix Decomposition
  • Sparse Priors


Dive into the research topics of 'Bayesian group factor analysis with structured sparsity'. Together they form a unique fingerprint.

Cite this