Universal estimation of erasure entropy

Jiming Yu; Sergio Verdu

doi:10.1109/TIT.2008.2008117

Universal estimation of erasure entropy

Jiming Yu, Sergio Verdu

Research output: Contribution to journal › Article › peer-review

4 Scopus citations

Abstract

Erasure entropy rate differs from Shannon's entropy rate in that the conditioning occurs with respect to both the past and the future, as opposed to only the past (or the future). In this paper, consistent universal algorithms for estimating erasure entropy rate are proposed based on the basic and extended context-tree weighting (CTW) algorithms. Simulation results for those algorithms applied to Markov sources, tree sources, and English texts are compared to those obtained by fixed-order plug-in estimators with different orders.

Original language	English (US)
Pages (from-to)	350-357
Number of pages	8
Journal	IEEE Transactions on Information Theory
Volume	55
Issue number	1
DOIs	https://doi.org/10.1109/TIT.2008.2008117
State	Published - Jan 22 2009

All Science Journal Classification (ASJC) codes

Information Systems
Computer Science Applications
Library and Information Sciences

Access to Document

10.1109/TIT.2008.2008117

Cite this

@article{15d31b1926e14bee8326bd5a8e11a2c1,

title = "Universal estimation of erasure entropy",

abstract = "Erasure entropy rate differs from Shannon's entropy rate in that the conditioning occurs with respect to both the past and the future, as opposed to only the past (or the future). In this paper, consistent universal algorithms for estimating erasure entropy rate are proposed based on the basic and extended context-tree weighting (CTW) algorithms. Simulation results for those algorithms applied to Markov sources, tree sources, and English texts are compared to those obtained by fixed-order plug-in estimators with different orders.",

author = "Jiming Yu and Sergio Verdu",

year = "2009",

month = jan,

day = "22",

doi = "10.1109/TIT.2008.2008117",

language = "English (US)",

volume = "55",

pages = "350--357",

journal = "IEEE Transactions on Information Theory",

issn = "0018-9448",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "1",

}

TY - JOUR

T1 - Universal estimation of erasure entropy

AU - Yu, Jiming

AU - Verdu, Sergio

PY - 2009/1/22

Y1 - 2009/1/22

N2 - Erasure entropy rate differs from Shannon's entropy rate in that the conditioning occurs with respect to both the past and the future, as opposed to only the past (or the future). In this paper, consistent universal algorithms for estimating erasure entropy rate are proposed based on the basic and extended context-tree weighting (CTW) algorithms. Simulation results for those algorithms applied to Markov sources, tree sources, and English texts are compared to those obtained by fixed-order plug-in estimators with different orders.

AB - Erasure entropy rate differs from Shannon's entropy rate in that the conditioning occurs with respect to both the past and the future, as opposed to only the past (or the future). In this paper, consistent universal algorithms for estimating erasure entropy rate are proposed based on the basic and extended context-tree weighting (CTW) algorithms. Simulation results for those algorithms applied to Markov sources, tree sources, and English texts are compared to those obtained by fixed-order plug-in estimators with different orders.

UR - http://www.scopus.com/inward/record.url?scp=57349093159&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=57349093159&partnerID=8YFLogxK

U2 - 10.1109/TIT.2008.2008117

DO - 10.1109/TIT.2008.2008117

M3 - Article

AN - SCOPUS:57349093159

SN - 0018-9448

VL - 55

SP - 350

EP - 357

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 1

ER -

Universal estimation of erasure entropy

Abstract

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this