Fast Moment Estimation for Generalized Latent Dirichlet Models

Shiwen Zhao, Barbara Engelhardt Martin, Sayan Mukherjee, David B. Dunson

Research output: Contribution to journalArticle

Abstract

We develop a generalized method of moments (GMM) approach for fast parameter estimation in a new class of Dirichlet latent variable models with mixed data types. Parameter estimation via GMM has computational and statistical advantages over alternative methods, such as expectation maximization, variational inference, and Markov chain Monte Carlo. A key computational advantage of our method, Moment Estimation for latent Dirichlet models (MELD), is that parameter estimation does not require instantiation of the latent variables. Moreover, performance is agnostic to distributional assumptions of the observations. We derive population moment conditions after marginalizing out the sample-specific Dirichlet latent variables. The moment conditions only depend on component mean parameters. We illustrate the utility of our approach on simulated data, comparing results from MELD to alternative methods, and we show the promise of our approach through the application to several datasets. Supplementary materials for this article are available online.

Original languageEnglish (US)
Pages (from-to)1528-1540
Number of pages13
JournalJournal of the American Statistical Association
Volume113
Issue number524
DOIs
StatePublished - Oct 2 2018

Fingerprint

Moment Estimation
Dirichlet
Parameter Estimation
Generalized Method of Moments
Moment Conditions
Latent Variables
Mixed Data
Latent Variable Models
Expectation Maximization
Alternatives
Markov Chain Monte Carlo
Model
Parameter estimation
Latent variables
Generalized method of moments
Moment conditions

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Zhao, Shiwen ; Engelhardt Martin, Barbara ; Mukherjee, Sayan ; Dunson, David B. / Fast Moment Estimation for Generalized Latent Dirichlet Models. In: Journal of the American Statistical Association. 2018 ; Vol. 113, No. 524. pp. 1528-1540.
@article{0f9fc0b96a314e0c9723053083454ddd,
title = "Fast Moment Estimation for Generalized Latent Dirichlet Models",
abstract = "We develop a generalized method of moments (GMM) approach for fast parameter estimation in a new class of Dirichlet latent variable models with mixed data types. Parameter estimation via GMM has computational and statistical advantages over alternative methods, such as expectation maximization, variational inference, and Markov chain Monte Carlo. A key computational advantage of our method, Moment Estimation for latent Dirichlet models (MELD), is that parameter estimation does not require instantiation of the latent variables. Moreover, performance is agnostic to distributional assumptions of the observations. We derive population moment conditions after marginalizing out the sample-specific Dirichlet latent variables. The moment conditions only depend on component mean parameters. We illustrate the utility of our approach on simulated data, comparing results from MELD to alternative methods, and we show the promise of our approach through the application to several datasets. Supplementary materials for this article are available online.",
author = "Shiwen Zhao and {Engelhardt Martin}, Barbara and Sayan Mukherjee and Dunson, {David B.}",
year = "2018",
month = "10",
day = "2",
doi = "10.1080/01621459.2017.1341839",
language = "English (US)",
volume = "113",
pages = "1528--1540",
journal = "Journal of the American Statistical Association",
issn = "0162-1459",
publisher = "Taylor and Francis Ltd.",
number = "524",

}

Fast Moment Estimation for Generalized Latent Dirichlet Models. / Zhao, Shiwen; Engelhardt Martin, Barbara; Mukherjee, Sayan; Dunson, David B.

In: Journal of the American Statistical Association, Vol. 113, No. 524, 02.10.2018, p. 1528-1540.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Fast Moment Estimation for Generalized Latent Dirichlet Models

AU - Zhao, Shiwen

AU - Engelhardt Martin, Barbara

AU - Mukherjee, Sayan

AU - Dunson, David B.

PY - 2018/10/2

Y1 - 2018/10/2

N2 - We develop a generalized method of moments (GMM) approach for fast parameter estimation in a new class of Dirichlet latent variable models with mixed data types. Parameter estimation via GMM has computational and statistical advantages over alternative methods, such as expectation maximization, variational inference, and Markov chain Monte Carlo. A key computational advantage of our method, Moment Estimation for latent Dirichlet models (MELD), is that parameter estimation does not require instantiation of the latent variables. Moreover, performance is agnostic to distributional assumptions of the observations. We derive population moment conditions after marginalizing out the sample-specific Dirichlet latent variables. The moment conditions only depend on component mean parameters. We illustrate the utility of our approach on simulated data, comparing results from MELD to alternative methods, and we show the promise of our approach through the application to several datasets. Supplementary materials for this article are available online.

AB - We develop a generalized method of moments (GMM) approach for fast parameter estimation in a new class of Dirichlet latent variable models with mixed data types. Parameter estimation via GMM has computational and statistical advantages over alternative methods, such as expectation maximization, variational inference, and Markov chain Monte Carlo. A key computational advantage of our method, Moment Estimation for latent Dirichlet models (MELD), is that parameter estimation does not require instantiation of the latent variables. Moreover, performance is agnostic to distributional assumptions of the observations. We derive population moment conditions after marginalizing out the sample-specific Dirichlet latent variables. The moment conditions only depend on component mean parameters. We illustrate the utility of our approach on simulated data, comparing results from MELD to alternative methods, and we show the promise of our approach through the application to several datasets. Supplementary materials for this article are available online.

UR - http://www.scopus.com/inward/record.url?scp=85057295962&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057295962&partnerID=8YFLogxK

U2 - 10.1080/01621459.2017.1341839

DO - 10.1080/01621459.2017.1341839

M3 - Article

VL - 113

SP - 1528

EP - 1540

JO - Journal of the American Statistical Association

JF - Journal of the American Statistical Association

SN - 0162-1459

IS - 524

ER -