Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy

Yoram Singer; Manfred K. Warmuth

Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy

Yoram Singer, Manfred K. Warmuth

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We describe a new iterative method for parameter estimation of Gaussian mixtures. The new method is based on a framework developed by Kivinen and Warmuth for supervised on-line learning. In contrast to gradient descent and EM, which estimate the mixture's covariance matrices, the proposed method estimates the inverses of the covariance matrices. Furthermore, the new parameter estimation procedure can be applied in both on-line and batch settings. We show experimentally that it is typically faster than EM, and usually requires about half as many iterations as EM.

Original language	English (US)
Title of host publication	Advances in Neural Information Processing Systems 11 - Proceedings of the 1998 Conference, NIPS 1998
Publisher	Neural information processing systems foundation
Pages	578-584
Number of pages	7
ISBN (Print)	0262112450, 9780262112451
State	Published - Jan 1 1999
Externally published	Yes
Event	12th Annual Conference on Neural Information Processing Systems, NIPS 1998 - Denver, CO, United States Duration: Nov 30 1998 → Dec 5 1998

Other

Other	12th Annual Conference on Neural Information Processing Systems, NIPS 1998
Country/Territory	United States
City	Denver, CO
Period	11/30/98 → 12/5/98

All Science Journal Classification (ASJC) codes

Computer Networks and Communications
Information Systems
Signal Processing

Cite this

@inproceedings{a46f60e22f024f0799cba3dba1692265,

title = "Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy",

abstract = "We describe a new iterative method for parameter estimation of Gaussian mixtures. The new method is based on a framework developed by Kivinen and Warmuth for supervised on-line learning. In contrast to gradient descent and EM, which estimate the mixture's covariance matrices, the proposed method estimates the inverses of the covariance matrices. Furthermore, the new parameter estimation procedure can be applied in both on-line and batch settings. We show experimentally that it is typically faster than EM, and usually requires about half as many iterations as EM.",

author = "Yoram Singer and Warmuth, {Manfred K.}",

year = "1999",

month = jan,

day = "1",

language = "English (US)",

isbn = "0262112450",

pages = "578--584",

booktitle = "Advances in Neural Information Processing Systems 11 - Proceedings of the 1998 Conference, NIPS 1998",

publisher = "Neural information processing systems foundation",

note = "12th Annual Conference on Neural Information Processing Systems, NIPS 1998 ; Conference date: 30-11-1998 Through 05-12-1998",

}

Singer, Y & Warmuth, MK 1999, Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy. in Advances in Neural Information Processing Systems 11 - Proceedings of the 1998 Conference, NIPS 1998. Neural information processing systems foundation, pp. 578-584, 12th Annual Conference on Neural Information Processing Systems, NIPS 1998, Denver, CO, United States, 11/30/98.

Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy. / Singer, Yoram; Warmuth, Manfred K.

Advances in Neural Information Processing Systems 11 - Proceedings of the 1998 Conference, NIPS 1998. Neural information processing systems foundation, 1999. p. 578-584.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy

AU - Singer, Yoram

AU - Warmuth, Manfred K.

PY - 1999/1/1

Y1 - 1999/1/1

N2 - We describe a new iterative method for parameter estimation of Gaussian mixtures. The new method is based on a framework developed by Kivinen and Warmuth for supervised on-line learning. In contrast to gradient descent and EM, which estimate the mixture's covariance matrices, the proposed method estimates the inverses of the covariance matrices. Furthermore, the new parameter estimation procedure can be applied in both on-line and batch settings. We show experimentally that it is typically faster than EM, and usually requires about half as many iterations as EM.

AB - We describe a new iterative method for parameter estimation of Gaussian mixtures. The new method is based on a framework developed by Kivinen and Warmuth for supervised on-line learning. In contrast to gradient descent and EM, which estimate the mixture's covariance matrices, the proposed method estimates the inverses of the covariance matrices. Furthermore, the new parameter estimation procedure can be applied in both on-line and batch settings. We show experimentally that it is typically faster than EM, and usually requires about half as many iterations as EM.

UR - http://www.scopus.com/inward/record.url?scp=0013397137&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0013397137&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:0013397137

SN - 0262112450

SN - 9780262112451

SP - 578

EP - 584

BT - Advances in Neural Information Processing Systems 11 - Proceedings of the 1998 Conference, NIPS 1998

PB - Neural information processing systems foundation

T2 - 12th Annual Conference on Neural Information Processing Systems, NIPS 1998

Y2 - 30 November 1998 through 5 December 1998

ER -

Batch and on-line parameter estimation of Gaussian mixtures based on the joint entropy

Abstract

Other

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this