TY - GEN
T1 - Universal erasure entropy estimation
AU - Yu, Jiming
AU - Verdú, Sergio
PY - 2006
Y1 - 2006
N2 - Erasure entropy rate (introduced recently by Verdú and Weissman) differs from Shannon's entropy rate in that the conditioning occurs with respect to both the past and the future, as opposed to only the past (or the future). In this paper, universal algorithms for estimating erasure entropy rate are proposed based on the basic and extended context-tree weighting (CTW) algorithms. Consistency results are shown for those CTW based algorithms. Simulation results for those algorithms applied to Markov sources, tree sources and English texts are compared to those obtained by fixed-order plug-in estimators with different orders. An estimate of the erasure entropy of English texts based on the proposed algorithms is about 0.22 bits per letter, which can be compared to an estimate of about 1.3 bits per letter for the entropy rate of English texts by a similar CTW based algorithm.
AB - Erasure entropy rate (introduced recently by Verdú and Weissman) differs from Shannon's entropy rate in that the conditioning occurs with respect to both the past and the future, as opposed to only the past (or the future). In this paper, universal algorithms for estimating erasure entropy rate are proposed based on the basic and extended context-tree weighting (CTW) algorithms. Consistency results are shown for those CTW based algorithms. Simulation results for those algorithms applied to Markov sources, tree sources and English texts are compared to those obtained by fixed-order plug-in estimators with different orders. An estimate of the erasure entropy of English texts based on the proposed algorithms is about 0.22 bits per letter, which can be compared to an estimate of about 1.3 bits per letter for the entropy rate of English texts by a similar CTW based algorithm.
UR - http://www.scopus.com/inward/record.url?scp=39049149881&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=39049149881&partnerID=8YFLogxK
U2 - 10.1109/ISIT.2006.262010
DO - 10.1109/ISIT.2006.262010
M3 - Conference contribution
AN - SCOPUS:39049149881
SN - 1424405041
SN - 9781424405046
T3 - IEEE International Symposium on Information Theory - Proceedings
SP - 2358
EP - 2362
BT - Proceedings - 2006 IEEE International Symposium on Information Theory, ISIT 2006
T2 - 2006 IEEE International Symposium on Information Theory, ISIT 2006
Y2 - 9 July 2006 through 14 July 2006
ER -