Functional properties of minimum mean-square error and mutual information

Yihong Wu; Sergio Verdú

doi:10.1109/TIT.2011.2174959

Functional properties of minimum mean-square error and mutual information

Yihong Wu, Sergio Verdú

Research output: Contribution to journal › Article › peer-review

100 Scopus citations

Abstract

In addition to exploring its various regularity properties, we show that the minimum mean-square error (MMSE) is a concave functional of the input-output joint distribution. In the case of additive Gaussian noise, the MMSE is shown to be weakly continuous in the input distribution and Lipschitz continuous with respect to the quadratic Wasserstein distance for peak-limited inputs. Regularity properties of mutual information are also obtained. Several applications to information theory and the central limit theorem are discussed.

Original language	English (US)
Article number	6084749
Pages (from-to)	1289-1301
Number of pages	13
Journal	IEEE Transactions on Information Theory
Volume	58
Issue number	3
DOIs	https://doi.org/10.1109/TIT.2011.2174959
State	Published - Mar 2012

All Science Journal Classification (ASJC) codes

Information Systems
Computer Science Applications
Library and Information Sciences

Keywords

Bayesian statistics
Gaussian noise
central limit theorem
minimum mean-square error (MMSE)
mutual information
non-Gaussian noise

Access to Document

10.1109/TIT.2011.2174959

Cite this

@article{a27d9af8ac8543109f3492d4c9a85d8f,

title = "Functional properties of minimum mean-square error and mutual information",

abstract = "In addition to exploring its various regularity properties, we show that the minimum mean-square error (MMSE) is a concave functional of the input-output joint distribution. In the case of additive Gaussian noise, the MMSE is shown to be weakly continuous in the input distribution and Lipschitz continuous with respect to the quadratic Wasserstein distance for peak-limited inputs. Regularity properties of mutual information are also obtained. Several applications to information theory and the central limit theorem are discussed.",

keywords = "Bayesian statistics, Gaussian noise, central limit theorem, minimum mean-square error (MMSE), mutual information, non-Gaussian noise",

author = "Yihong Wu and Sergio Verd{\'u}",

year = "2012",

month = mar,

doi = "10.1109/TIT.2011.2174959",

language = "English (US)",

volume = "58",

pages = "1289--1301",

journal = "IEEE Transactions on Information Theory",

issn = "0018-9448",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "3",

}

TY - JOUR

T1 - Functional properties of minimum mean-square error and mutual information

AU - Wu, Yihong

AU - Verdú, Sergio

PY - 2012/3

Y1 - 2012/3

N2 - In addition to exploring its various regularity properties, we show that the minimum mean-square error (MMSE) is a concave functional of the input-output joint distribution. In the case of additive Gaussian noise, the MMSE is shown to be weakly continuous in the input distribution and Lipschitz continuous with respect to the quadratic Wasserstein distance for peak-limited inputs. Regularity properties of mutual information are also obtained. Several applications to information theory and the central limit theorem are discussed.

AB - In addition to exploring its various regularity properties, we show that the minimum mean-square error (MMSE) is a concave functional of the input-output joint distribution. In the case of additive Gaussian noise, the MMSE is shown to be weakly continuous in the input distribution and Lipschitz continuous with respect to the quadratic Wasserstein distance for peak-limited inputs. Regularity properties of mutual information are also obtained. Several applications to information theory and the central limit theorem are discussed.

KW - Bayesian statistics

KW - Gaussian noise

KW - central limit theorem

KW - minimum mean-square error (MMSE)

KW - mutual information

KW - non-Gaussian noise

UR - http://www.scopus.com/inward/record.url?scp=84863264784&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863264784&partnerID=8YFLogxK

U2 - 10.1109/TIT.2011.2174959

DO - 10.1109/TIT.2011.2174959

M3 - Article

AN - SCOPUS:84863264784

SN - 0018-9448

VL - 58

SP - 1289

EP - 1301

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 3

M1 - 6084749

ER -

Functional properties of minimum mean-square error and mutual information

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this