Empirical distribution of good channel codes with nonvanishing error probability

Yury Polyanskiy, Sergio Verdu

Research output: Contribution to journalArticlepeer-review

32 Scopus citations


This paper studies several properties of channel codes that approach the fundamental limits of a given (discrete or Gaussian) memoryless channel with a nonvanishing probability of error. The output distribution induced by an epsilon -capacity-achieving code is shown to be close in a strong sense to the capacity achieving output distribution. Relying on the concentration of measure (isoperimetry) property enjoyed by the latter, it is shown that regular (Lipschitz) functions of channel outputs can be precisely estimated and turn out to be essentially nonrandom and independent of the actual code. It is also shown that the output distribution of a good code and the capacity achieving one cannot be distinguished with exponential reliability. The random process produced at the output of the channel is shown to satisfy the asymptotic equipartition property.

Original languageEnglish (US)
Article number6620929
Pages (from-to)5-21
Number of pages17
JournalIEEE Transactions on Information Theory
Issue number1
StatePublished - Jan 2014

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences


  • Additive white Gaussian noise
  • Shannon theory
  • asymptotic equipartition property
  • concentration of measure
  • discrete memoryless channels
  • empirical output statistics
  • relative entropy


Dive into the research topics of 'Empirical distribution of good channel codes with nonvanishing error probability'. Together they form a unique fingerprint.

Cite this