TY - GEN

T1 - A new entropy power inequality for integer-valued random variables

AU - Haghighatshoar, Saeid

AU - Abbe, Emmanuel

AU - Telatar, Emre

PY - 2013/12/19

Y1 - 2013/12/19

N2 - The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy, but no universal inequality is known (beyond trivial ones). More recently, the sumset theory for the entropy function yields a sharp inequality H(X + X') - H(X) ≥ 1/2 - o(l) when Χ,Χ' are i.i.d. with high entropy. This paper provides the inequality H(X + X') - H(X) ≥ g(H(X)), where X, X' are arbitrary i.i.d. integer-valued random variables and where g is a universal strictly positive function on R+ satisfying g(0) = 0. Extensions to non identically distributed random variables and to conditional entropies are also obtained.

AB - The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy, but no universal inequality is known (beyond trivial ones). More recently, the sumset theory for the entropy function yields a sharp inequality H(X + X') - H(X) ≥ 1/2 - o(l) when Χ,Χ' are i.i.d. with high entropy. This paper provides the inequality H(X + X') - H(X) ≥ g(H(X)), where X, X' are arbitrary i.i.d. integer-valued random variables and where g is a universal strictly positive function on R+ satisfying g(0) = 0. Extensions to non identically distributed random variables and to conditional entropies are also obtained.

KW - Entropic inequalities

KW - Entropy power inequality

KW - Mrs. Gerber's lemma

KW - Shannon sumset theory

UR - http://www.scopus.com/inward/record.url?scp=84890321615&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84890321615&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2013.6620294

DO - 10.1109/ISIT.2013.6620294

M3 - Conference contribution

AN - SCOPUS:84890321615

SN - 9781479904464

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 589

EP - 593

BT - 2013 IEEE International Symposium on Information Theory, ISIT 2013

T2 - 2013 IEEE International Symposium on Information Theory, ISIT 2013

Y2 - 7 July 2013 through 12 July 2013

ER -