Learning continuous distributions: Simulations with field theoretic priors

Ilya Nemenman, William Bialek

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Learning of a smooth but nonparametric probability density can be regularized using methods of Quantum Field Theory. We implement a field theoretic prior numerically, test its efficacy, and show that the free parameter of the theory ('smoothness scale') can be determined self consistently by the data; this forms an infinite dimensional generalization of the MDL principle. Finally, we study the implications of one's choice of the prior and the parameterization and conclude that the smoothness scale determination makes density estimation very weakly sensitive to the choice of the prior, and that even wrong choices can be advantageous for small data sets.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 13 - Proceedings of the 2000 Conference, NIPS 2000
PublisherNeural information processing systems foundation
ISBN (Print)0262122413, 9780262122412
StatePublished - Jan 1 2001
Event14th Annual Neural Information Processing Systems Conference, NIPS 2000 - Denver, CO, United States
Duration: Nov 27 2000Dec 2 2000

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258

Other

Other14th Annual Neural Information Processing Systems Conference, NIPS 2000
CountryUnited States
CityDenver, CO
Period11/27/0012/2/00

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Fingerprint Dive into the research topics of 'Learning continuous distributions: Simulations with field theoretic priors'. Together they form a unique fingerprint.

  • Cite this

    Nemenman, I., & Bialek, W. (2001). Learning continuous distributions: Simulations with field theoretic priors. In Advances in Neural Information Processing Systems 13 - Proceedings of the 2000 Conference, NIPS 2000 (Advances in Neural Information Processing Systems). Neural information processing systems foundation.