TY - JOUR

T1 - Conditional Mean Estimation in Gaussian Noise

T2 - A Meta Derivative Identity With Applications

AU - Dytso, Alex

AU - Poor, H. Vincent

AU - Shamai Shitz, Shlomo

N1 - Funding Information:
This work was supported in part by the U.S. National Science Foundation under Grant CCF-1908308 and in part by the United States-Israel Binational Science Foundation under Grant BSF-2018710.
Publisher Copyright:
© 1963-2012 IEEE.

PY - 2023/3/1

Y1 - 2023/3/1

N2 - Consider a channel Y= X+ N where X is an n -dimensional random vector, and N is a multivariate Gaussian vector with a full-rank covariance matrix KN. The object under consideration in this paper is the conditional mean of X given Y=y , that is {E} XY=y. Several identities in the literature connect EXY=y to other quantities such as the conditional variance, score functions, and higher-order conditional moments. The objective of this paper is to provide a unifying view of these identities. In the first part of the paper, a general derivative identity for the conditional mean estimator is derived. Specifically, for the Markov chain U {X} {Y}, it is shown that the Jacobian matrix of {E}[{U}- {Y}=y is given by K {N}-1} {{Cov}} {X},{U}-{Y}=y where {Cov}} ({X},{U}- {Y}={y) is the conditional covariance. In the second part of the paper, via various choices of the random vector {U} , the new identity is used to recover and generalize many of the known identities and derive some new identities. First, a simple proof of the Hatsel and Nolte identity for the conditional variance is shown. Second, a simple proof of the recursive identity due to Jaffer is provided. The Jaffer identity is then further explored, and several equivalent statements are derived, such as an identity for the higher-order conditional expectation (i.e., {E}[{X/k}|{Y} ) in terms of the derivatives of the conditional expectation. Third, a new fundamental connection between the conditional cumulants and the conditional expectation is demonstrated. In particular, in the univariate case, it is shown that the k -th derivative of the conditional expectation is proportional to the (k+1) -th conditional cumulant. A similar expression is derived in the multivariate case.

AB - Consider a channel Y= X+ N where X is an n -dimensional random vector, and N is a multivariate Gaussian vector with a full-rank covariance matrix KN. The object under consideration in this paper is the conditional mean of X given Y=y , that is {E} XY=y. Several identities in the literature connect EXY=y to other quantities such as the conditional variance, score functions, and higher-order conditional moments. The objective of this paper is to provide a unifying view of these identities. In the first part of the paper, a general derivative identity for the conditional mean estimator is derived. Specifically, for the Markov chain U {X} {Y}, it is shown that the Jacobian matrix of {E}[{U}- {Y}=y is given by K {N}-1} {{Cov}} {X},{U}-{Y}=y where {Cov}} ({X},{U}- {Y}={y) is the conditional covariance. In the second part of the paper, via various choices of the random vector {U} , the new identity is used to recover and generalize many of the known identities and derive some new identities. First, a simple proof of the Hatsel and Nolte identity for the conditional variance is shown. Second, a simple proof of the recursive identity due to Jaffer is provided. The Jaffer identity is then further explored, and several equivalent statements are derived, such as an identity for the higher-order conditional expectation (i.e., {E}[{X/k}|{Y} ) in terms of the derivatives of the conditional expectation. Third, a new fundamental connection between the conditional cumulants and the conditional expectation is demonstrated. In particular, in the univariate case, it is shown that the k -th derivative of the conditional expectation is proportional to the (k+1) -th conditional cumulant. A similar expression is derived in the multivariate case.

KW - Vector Gaussian noise

KW - conditional cumulant

KW - conditional mean estimator

KW - minimum mean squared error

UR - http://www.scopus.com/inward/record.url?scp=85140790930&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85140790930&partnerID=8YFLogxK

U2 - 10.1109/TIT.2022.3216012

DO - 10.1109/TIT.2022.3216012

M3 - Article

AN - SCOPUS:85140790930

SN - 0018-9448

VL - 69

SP - 1883

EP - 1898

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 3

ER -