Relative entropy at the channel output of a capacity-achieving code

Yury Polyanskiy; Sergio Verdú

doi:10.1109/Allerton.2011.6120149

Relative entropy at the channel output of a capacity-achieving code

Yury Polyanskiy, Sergio Verdú

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

4 Scopus citations

Abstract

In this paper we establish a new inequality tying together the coding rate, the probability of error and the relative entropy between the channel and the auxiliary output distribution. This inequality is then used to show the strong converse, and to prove that the output distribution of a code must be close, in relative entropy, to the capacity achieving output distribution (for DMC and AWGN). One of the key tools in our analysis is the concentration of measure (isoperimetry).

Original language	English (US)
Title of host publication	2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011
Pages	52-59
Number of pages	8
DOIs	https://doi.org/10.1109/Allerton.2011.6120149
State	Published - 2011
Event	2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011 - Monticello, IL, United States Duration: Sep 28 2011 → Sep 30 2011

Publication series

Name	2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011

Other

Other	2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011
Country/Territory	United States
City	Monticello, IL
Period	9/28/11 → 9/30/11

All Science Journal Classification (ASJC) codes

Computer Networks and Communications
Control and Systems Engineering

Keywords

Shannon theory
additive white Gaussian noise
concentration of measure
discrete memoryless channels
empirical output statistics
general channels
information measures
strong converse

Access to Document

10.1109/Allerton.2011.6120149

Cite this

Polyanskiy, Y., & Verdú, S. (2011). Relative entropy at the channel output of a capacity-achieving code. In 2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011 (pp. 52-59). Article 6120149 (2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011). https://doi.org/10.1109/Allerton.2011.6120149

@inproceedings{11a30bfe012941329a4870397a0e7760,

title = "Relative entropy at the channel output of a capacity-achieving code",

abstract = "In this paper we establish a new inequality tying together the coding rate, the probability of error and the relative entropy between the channel and the auxiliary output distribution. This inequality is then used to show the strong converse, and to prove that the output distribution of a code must be close, in relative entropy, to the capacity achieving output distribution (for DMC and AWGN). One of the key tools in our analysis is the concentration of measure (isoperimetry).",

keywords = "Shannon theory, additive white Gaussian noise, concentration of measure, discrete memoryless channels, empirical output statistics, general channels, information measures, strong converse",

author = "Yury Polyanskiy and Sergio Verd{\'u}",

year = "2011",

doi = "10.1109/Allerton.2011.6120149",

language = "English (US)",

isbn = "9781457718168",

series = "2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011",

pages = "52--59",

booktitle = "2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011",

note = "2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011 ; Conference date: 28-09-2011 Through 30-09-2011",

}

Polyanskiy, Y & Verdú, S 2011, Relative entropy at the channel output of a capacity-achieving code. in 2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011., 6120149, 2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011, pp. 52-59, 2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011, Monticello, IL, United States, 9/28/11. https://doi.org/10.1109/Allerton.2011.6120149

Relative entropy at the channel output of a capacity-achieving code. / Polyanskiy, Yury; Verdú, Sergio.
2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011. 2011. p. 52-59 6120149 (2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Relative entropy at the channel output of a capacity-achieving code

AU - Polyanskiy, Yury

AU - Verdú, Sergio

PY - 2011

Y1 - 2011

N2 - In this paper we establish a new inequality tying together the coding rate, the probability of error and the relative entropy between the channel and the auxiliary output distribution. This inequality is then used to show the strong converse, and to prove that the output distribution of a code must be close, in relative entropy, to the capacity achieving output distribution (for DMC and AWGN). One of the key tools in our analysis is the concentration of measure (isoperimetry).

AB - In this paper we establish a new inequality tying together the coding rate, the probability of error and the relative entropy between the channel and the auxiliary output distribution. This inequality is then used to show the strong converse, and to prove that the output distribution of a code must be close, in relative entropy, to the capacity achieving output distribution (for DMC and AWGN). One of the key tools in our analysis is the concentration of measure (isoperimetry).

KW - Shannon theory

KW - additive white Gaussian noise

KW - concentration of measure

KW - discrete memoryless channels

KW - empirical output statistics

KW - general channels

KW - information measures

KW - strong converse

UR - http://www.scopus.com/inward/record.url?scp=84856110936&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84856110936&partnerID=8YFLogxK

U2 - 10.1109/Allerton.2011.6120149

DO - 10.1109/Allerton.2011.6120149

M3 - Conference contribution

AN - SCOPUS:84856110936

SN - 9781457718168

T3 - 2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011

SP - 52

EP - 59

BT - 2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011

T2 - 2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011

Y2 - 28 September 2011 through 30 September 2011

ER -

Relative entropy at the channel output of a capacity-achieving code

Abstract

Publication series

Other

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this