TY - JOUR

T1 - On the minimum mean $p$ th error in Gaussian noise channels and its applications

AU - Dytso, Alex

AU - Bustin, Ronit

AU - Tuninetti, Daniela

AU - Devroye, Natasha

AU - Poor, H. Vincent

AU - Shitz, Shlomo Shamai

N1 - Funding Information:
Manuscript received July 5, 2016; revised August 7, 2017; accepted November 18, 2017. Date of publication December 13, 2017; date of current version February 15, 2018. A. Dytso was supported in part by the U.S. National Science Foundation (NSF) under Grants CNS-1456793, CCF-1420575, and CCF-1422511. R. Bustin was supported in part by the Women Postdoctoral Scholarship of Israel’s Council for Higher Education (VATAT) 2014-2015 and in part by Union’s Horizon 2020 Research and Innovation Programme (2016) under Grant 694630. D. Tuninetti and N. Devroye were supported by the NSF under Grant CCF-1422511. H. V. Poor was supported by the NSF under Grant CNS-1456793 and Grant CCF-1420575. The work of S. Shamai (Shitz) was supported in part by the Israel Science Foundation, in part by the S. and N. Grand Research Fund, and in part by the Union’s Horizon 2020 Research and Innovation Programme under Grant 694630. This paper was presented in part at the 2016 IEEE Information Theory Workshop and in part at the 2016 IEEE International Symposium on Information Theory.
Publisher Copyright:
© 1963-2012 IEEE.

PY - 2018/3

Y1 - 2018/3

N2 - The problem of estimating an arbitrary random vector from its observation corrupted by additive white Gaussian noise, where the cost function is taken to be the minimum mean $p$ th error (MMPE), is considered. The classical minimum mean square error (MMSE) is a special case of the MMPE. Several bounds, properties, and applications of the MMPE are derived and discussed. The optimal MMPE estimator is found for Gaussian and binary input distributions. Properties of the MMPE as a function of the input distribution, signal-To-noise-ratio (SNR) and order $p$ are derived. The 'single-crossing-point property' (SCPP) which provides an upper bound on the MMSE, and which together with the mutual information-MMSE relationship is a powerful tool in deriving converse proofs in multi-user information theory, is extended to the MMPE. Moreover, a complementary bound to the SCPP is derived. As a first application of the MMPE, a bound on the conditional differential entropy in terms of the MMPE is provided, which then yields a generalization of the Ozarow-Wyner lower bound on the mutual information achieved by a discrete input on a Gaussian noise channel. As a second application, the MMPE is shown to improve on previous characterizations of the phase transition phenomenon that manifests, in the limit as the length of the capacity achieving code goes to infinity, as a discontinuity of the MMSE as a function of SNR. As a final application, the MMPE is used to show new bounds on the second derivative of mutual information, or the first derivative of the MMSE.

AB - The problem of estimating an arbitrary random vector from its observation corrupted by additive white Gaussian noise, where the cost function is taken to be the minimum mean $p$ th error (MMPE), is considered. The classical minimum mean square error (MMSE) is a special case of the MMPE. Several bounds, properties, and applications of the MMPE are derived and discussed. The optimal MMPE estimator is found for Gaussian and binary input distributions. Properties of the MMPE as a function of the input distribution, signal-To-noise-ratio (SNR) and order $p$ are derived. The 'single-crossing-point property' (SCPP) which provides an upper bound on the MMSE, and which together with the mutual information-MMSE relationship is a powerful tool in deriving converse proofs in multi-user information theory, is extended to the MMPE. Moreover, a complementary bound to the SCPP is derived. As a first application of the MMPE, a bound on the conditional differential entropy in terms of the MMPE is provided, which then yields a generalization of the Ozarow-Wyner lower bound on the mutual information achieved by a discrete input on a Gaussian noise channel. As a second application, the MMPE is shown to improve on previous characterizations of the phase transition phenomenon that manifests, in the limit as the length of the capacity achieving code goes to infinity, as a discontinuity of the MMSE as a function of SNR. As a final application, the MMPE is used to show new bounds on the second derivative of mutual information, or the first derivative of the MMSE.

KW - I-MMSE

KW - estimation

UR - http://www.scopus.com/inward/record.url?scp=85038821005&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038821005&partnerID=8YFLogxK

U2 - 10.1109/TIT.2017.2782786

DO - 10.1109/TIT.2017.2782786

M3 - Article

AN - SCOPUS:85038821005

SN - 0018-9448

VL - 64

SP - 2012

EP - 2037

JO - IRE Professional Group on Information Theory

JF - IRE Professional Group on Information Theory

IS - 3

ER -