TY - JOUR

T1 - On the minimum mean $p$ th error in Gaussian noise channels and its applications

AU - Dytso, Alex

AU - Bustin, Ronit

AU - Tuninetti, Daniela

AU - Devroye, Natasha

AU - Poor, H. Vincent

AU - Shitz, Shlomo Shamai

N1 - Publisher Copyright:
© 1963-2012 IEEE.

PY - 2018/3

Y1 - 2018/3

N2 - The problem of estimating an arbitrary random vector from its observation corrupted by additive white Gaussian noise, where the cost function is taken to be the minimum mean $p$ th error (MMPE), is considered. The classical minimum mean square error (MMSE) is a special case of the MMPE. Several bounds, properties, and applications of the MMPE are derived and discussed. The optimal MMPE estimator is found for Gaussian and binary input distributions. Properties of the MMPE as a function of the input distribution, signal-To-noise-ratio (SNR) and order $p$ are derived. The 'single-crossing-point property' (SCPP) which provides an upper bound on the MMSE, and which together with the mutual information-MMSE relationship is a powerful tool in deriving converse proofs in multi-user information theory, is extended to the MMPE. Moreover, a complementary bound to the SCPP is derived. As a first application of the MMPE, a bound on the conditional differential entropy in terms of the MMPE is provided, which then yields a generalization of the Ozarow-Wyner lower bound on the mutual information achieved by a discrete input on a Gaussian noise channel. As a second application, the MMPE is shown to improve on previous characterizations of the phase transition phenomenon that manifests, in the limit as the length of the capacity achieving code goes to infinity, as a discontinuity of the MMSE as a function of SNR. As a final application, the MMPE is used to show new bounds on the second derivative of mutual information, or the first derivative of the MMSE.

AB - The problem of estimating an arbitrary random vector from its observation corrupted by additive white Gaussian noise, where the cost function is taken to be the minimum mean $p$ th error (MMPE), is considered. The classical minimum mean square error (MMSE) is a special case of the MMPE. Several bounds, properties, and applications of the MMPE are derived and discussed. The optimal MMPE estimator is found for Gaussian and binary input distributions. Properties of the MMPE as a function of the input distribution, signal-To-noise-ratio (SNR) and order $p$ are derived. The 'single-crossing-point property' (SCPP) which provides an upper bound on the MMSE, and which together with the mutual information-MMSE relationship is a powerful tool in deriving converse proofs in multi-user information theory, is extended to the MMPE. Moreover, a complementary bound to the SCPP is derived. As a first application of the MMPE, a bound on the conditional differential entropy in terms of the MMPE is provided, which then yields a generalization of the Ozarow-Wyner lower bound on the mutual information achieved by a discrete input on a Gaussian noise channel. As a second application, the MMPE is shown to improve on previous characterizations of the phase transition phenomenon that manifests, in the limit as the length of the capacity achieving code goes to infinity, as a discontinuity of the MMSE as a function of SNR. As a final application, the MMPE is used to show new bounds on the second derivative of mutual information, or the first derivative of the MMSE.

KW - I-MMSE

KW - estimation

UR - http://www.scopus.com/inward/record.url?scp=85038821005&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038821005&partnerID=8YFLogxK

U2 - 10.1109/TIT.2017.2782786

DO - 10.1109/TIT.2017.2782786

M3 - Article

AN - SCOPUS:85038821005

SN - 0018-9448

VL - 64

SP - 2012

EP - 2037

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 3

ER -