TY - GEN

T1 - Conditional entropy and error probability

AU - Ho, Siu Wai

AU - Verdu, Sergio

PY - 2008/9/29

Y1 - 2008/9/29

N2 - Fano's inequality relates the error probability and conditional entropy of a finitely-valued random variable X given another random variable Y. It is not necessarily tight when the marginal distribution of X is fixed. In this paper, we consider both finite and countably infinite alphabets. A tight upper bound on the conditional entropy of X given Y is given in terms of the error probability and the marginal distribution of X. A new lower bound on the conditional entropy for countably infinite alphabet is also found. The equivalence of the reliability criteria of vanishing error probability and vanishing conditional entropy is established in wide generality.

AB - Fano's inequality relates the error probability and conditional entropy of a finitely-valued random variable X given another random variable Y. It is not necessarily tight when the marginal distribution of X is fixed. In this paper, we consider both finite and countably infinite alphabets. A tight upper bound on the conditional entropy of X given Y is given in terms of the error probability and the marginal distribution of X. A new lower bound on the conditional entropy for countably infinite alphabet is also found. The equivalence of the reliability criteria of vanishing error probability and vanishing conditional entropy is established in wide generality.

UR - http://www.scopus.com/inward/record.url?scp=52349118909&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=52349118909&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2008.4595262

DO - 10.1109/ISIT.2008.4595262

M3 - Conference contribution

AN - SCOPUS:52349118909

SN - 9781424422579

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 1622

EP - 1626

BT - Proceedings - 2008 IEEE International Symposium on Information Theory, ISIT 2008

T2 - 2008 IEEE International Symposium on Information Theory, ISIT 2008

Y2 - 6 July 2008 through 11 July 2008

ER -