TY - GEN
T1 - Conditional entropy and error probability
AU - Ho, Siu Wai
AU - Verdú, Sergio
PY - 2008
Y1 - 2008
N2 - Fano's inequality relates the error probability and conditional entropy of a finitely-valued random variable X given another random variable Y. It is not necessarily tight when the marginal distribution of X is fixed. In this paper, we consider both finite and countably infinite alphabets. A tight upper bound on the conditional entropy of X given Y is given in terms of the error probability and the marginal distribution of X. A new lower bound on the conditional entropy for countably infinite alphabet is also found. The equivalence of the reliability criteria of vanishing error probability and vanishing conditional entropy is established in wide generality.
AB - Fano's inequality relates the error probability and conditional entropy of a finitely-valued random variable X given another random variable Y. It is not necessarily tight when the marginal distribution of X is fixed. In this paper, we consider both finite and countably infinite alphabets. A tight upper bound on the conditional entropy of X given Y is given in terms of the error probability and the marginal distribution of X. A new lower bound on the conditional entropy for countably infinite alphabet is also found. The equivalence of the reliability criteria of vanishing error probability and vanishing conditional entropy is established in wide generality.
UR - http://www.scopus.com/inward/record.url?scp=52349118909&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=52349118909&partnerID=8YFLogxK
U2 - 10.1109/ISIT.2008.4595262
DO - 10.1109/ISIT.2008.4595262
M3 - Conference contribution
AN - SCOPUS:52349118909
SN - 9781424422579
T3 - IEEE International Symposium on Information Theory - Proceedings
SP - 1622
EP - 1626
BT - Proceedings - 2008 IEEE International Symposium on Information Theory, ISIT 2008
T2 - 2008 IEEE International Symposium on Information Theory, ISIT 2008
Y2 - 6 July 2008 through 11 July 2008
ER -