A new entropy power inequality for integer-valued random variables

Saeid Haghighatshoar, Emmanuel Abbe, Emre Telatar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy, but no universal inequality is known (beyond trivial ones). More recently, the sumset theory for the entropy function yields a sharp inequality H(X + X') - H(X) ≥ 1/2 - o(l) when Χ,Χ' are i.i.d. with high entropy. This paper provides the inequality H(X + X') - H(X) ≥ g(H(X)), where X, X' are arbitrary i.i.d. integer-valued random variables and where g is a universal strictly positive function on R+ satisfying g(0) = 0. Extensions to non identically distributed random variables and to conditional entropies are also obtained.

Original languageEnglish (US)
Title of host publication2013 IEEE International Symposium on Information Theory, ISIT 2013
Pages589-593
Number of pages5
DOIs
StatePublished - Dec 19 2013
Event2013 IEEE International Symposium on Information Theory, ISIT 2013 - Istanbul, Turkey
Duration: Jul 7 2013Jul 12 2013

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
ISSN (Print)2157-8095

Other

Other2013 IEEE International Symposium on Information Theory, ISIT 2013
CountryTurkey
CityIstanbul
Period7/7/137/12/13

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics

Keywords

  • Entropic inequalities
  • Entropy power inequality
  • Mrs. Gerber's lemma
  • Shannon sumset theory

Fingerprint Dive into the research topics of 'A new entropy power inequality for integer-valued random variables'. Together they form a unique fingerprint.

  • Cite this

    Haghighatshoar, S., Abbe, E., & Telatar, E. (2013). A new entropy power inequality for integer-valued random variables. In 2013 IEEE International Symposium on Information Theory, ISIT 2013 (pp. 589-593). [6620294] (IEEE International Symposium on Information Theory - Proceedings). https://doi.org/10.1109/ISIT.2013.6620294