Universal estimation of divergence for continuous distributions via data-dependent partitions

Qing Wang, Sanjeev R. Kulkarni, Sergio Verdú

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

We present a universal estimator of the divergence D(P||Q) for two arbitrary continuous distributions P and Q satisfying certain regularity conditions. This algorithm, which observes i.i.d. samples from both P and Q, is based on the estimation of the Radon-Nikodym derivative ^ via a datadependent partition of the observation space. Strong convergence of this estimator is proved with an empirically equivalent segmentation of the space. This basic estimator is further improved by adaptive partitioning schemes and by bias correction. In the simulations, we compare our estimators with the plug-in estimator and estimators based on other partitioning approaches. Experimental results show that our methods achieve the best convergence performance in most of the tested cases.

Original languageEnglish (US)
Title of host publicationProceedings of the 2005 IEEE International Symposium on Information Theory, ISIT 05
Pages152-156
Number of pages5
DOIs
StatePublished - Dec 1 2005
Event2005 IEEE International Symposium on Information Theory, ISIT 05 - Adelaide, Australia
Duration: Sep 4 2005Sep 9 2005

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
Volume2005
ISSN (Print)2157-8099

Other

Other2005 IEEE International Symposium on Information Theory, ISIT 05
CountryAustralia
CityAdelaide
Period9/4/059/9/05

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Universal estimation of divergence for continuous distributions via data-dependent partitions'. Together they form a unique fingerprint.

  • Cite this

    Wang, Q., Kulkarni, S. R., & Verdú, S. (2005). Universal estimation of divergence for continuous distributions via data-dependent partitions. In Proceedings of the 2005 IEEE International Symposium on Information Theory, ISIT 05 (pp. 152-156). [1523312] (IEEE International Symposium on Information Theory - Proceedings; Vol. 2005). https://doi.org/10.1109/ISIT.2005.1523312