TY - GEN
T1 - A new entropy power inequality for integer-valued random variables
AU - Haghighatshoar, Saeid
AU - Abbe, Emmanuel
AU - Telatar, Emre
PY - 2013
Y1 - 2013
N2 - The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy, but no universal inequality is known (beyond trivial ones). More recently, the sumset theory for the entropy function yields a sharp inequality H(X + X') - H(X) ≥ 1/2 - o(l) when Χ,Χ' are i.i.d. with high entropy. This paper provides the inequality H(X + X') - H(X) ≥ g(H(X)), where X, X' are arbitrary i.i.d. integer-valued random variables and where g is a universal strictly positive function on R+ satisfying g(0) = 0. Extensions to non identically distributed random variables and to conditional entropies are also obtained.
AB - The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy, but no universal inequality is known (beyond trivial ones). More recently, the sumset theory for the entropy function yields a sharp inequality H(X + X') - H(X) ≥ 1/2 - o(l) when Χ,Χ' are i.i.d. with high entropy. This paper provides the inequality H(X + X') - H(X) ≥ g(H(X)), where X, X' are arbitrary i.i.d. integer-valued random variables and where g is a universal strictly positive function on R+ satisfying g(0) = 0. Extensions to non identically distributed random variables and to conditional entropies are also obtained.
KW - Entropic inequalities
KW - Entropy power inequality
KW - Mrs. Gerber's lemma
KW - Shannon sumset theory
UR - http://www.scopus.com/inward/record.url?scp=84890321615&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84890321615&partnerID=8YFLogxK
U2 - 10.1109/ISIT.2013.6620294
DO - 10.1109/ISIT.2013.6620294
M3 - Conference contribution
AN - SCOPUS:84890321615
SN - 9781479904464
T3 - IEEE International Symposium on Information Theory - Proceedings
SP - 589
EP - 593
BT - 2013 IEEE International Symposium on Information Theory, ISIT 2013
T2 - 2013 IEEE International Symposium on Information Theory, ISIT 2013
Y2 - 7 July 2013 through 12 July 2013
ER -