TY - GEN
T1 - Learning continuous distributions
T2 - 14th Annual Neural Information Processing Systems Conference, NIPS 2000
AU - Nemenman, Ilya
AU - Bialek, William
PY - 2001
Y1 - 2001
N2 - Learning of a smooth but nonparametric probability density can be regularized using methods of Quantum Field Theory. We implement a field theoretic prior numerically, test its efficacy, and show that the free parameter of the theory ('smoothness scale') can be determined self consistently by the data; this forms an infinite dimensional generalization of the MDL principle. Finally, we study the implications of one's choice of the prior and the parameterization and conclude that the smoothness scale determination makes density estimation very weakly sensitive to the choice of the prior, and that even wrong choices can be advantageous for small data sets.
AB - Learning of a smooth but nonparametric probability density can be regularized using methods of Quantum Field Theory. We implement a field theoretic prior numerically, test its efficacy, and show that the free parameter of the theory ('smoothness scale') can be determined self consistently by the data; this forms an infinite dimensional generalization of the MDL principle. Finally, we study the implications of one's choice of the prior and the parameterization and conclude that the smoothness scale determination makes density estimation very weakly sensitive to the choice of the prior, and that even wrong choices can be advantageous for small data sets.
UR - http://www.scopus.com/inward/record.url?scp=0039214381&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0039214381&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:0039214381
SN - 0262122413
SN - 9780262122412
T3 - Advances in Neural Information Processing Systems
BT - Advances in Neural Information Processing Systems 13 - Proceedings of the 2000 Conference, NIPS 2000
PB - Neural information processing systems foundation
Y2 - 27 November 2000 through 2 December 2000
ER -