Universal erasure entropy estimation

Jiming Yu, Sergio Verdu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

Erasure entropy rate (introduced recently by Verdú and Weissman) differs from Shannon's entropy rate in that the conditioning occurs with respect to both the past and the future, as opposed to only the past (or the future). In this paper, universal algorithms for estimating erasure entropy rate are proposed based on the basic and extended context-tree weighting (CTW) algorithms. Consistency results are shown for those CTW based algorithms. Simulation results for those algorithms applied to Markov sources, tree sources and English texts are compared to those obtained by fixed-order plug-in estimators with different orders. An estimate of the erasure entropy of English texts based on the proposed algorithms is about 0.22 bits per letter, which can be compared to an estimate of about 1.3 bits per letter for the entropy rate of English texts by a similar CTW based algorithm.

Original languageEnglish (US)
Title of host publicationProceedings - 2006 IEEE International Symposium on Information Theory, ISIT 2006
Pages2358-2362
Number of pages5
DOIs
StatePublished - Dec 1 2006
Event2006 IEEE International Symposium on Information Theory, ISIT 2006 - Seattle, WA, United States
Duration: Jul 9 2006Jul 14 2006

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
ISSN (Print)2157-8101

Other

Other2006 IEEE International Symposium on Information Theory, ISIT 2006
CountryUnited States
CitySeattle, WA
Period7/9/067/14/06

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Universal erasure entropy estimation'. Together they form a unique fingerprint.

  • Cite this

    Yu, J., & Verdu, S. (2006). Universal erasure entropy estimation. In Proceedings - 2006 IEEE International Symposium on Information Theory, ISIT 2006 (pp. 2358-2362). [4036392] (IEEE International Symposium on Information Theory - Proceedings). https://doi.org/10.1109/ISIT.2006.262010