TY - GEN

T1 - Mismatched estimation and relative entropy

AU - Verdú, Sergio

PY - 2009/11/19

Y1 - 2009/11/19

N2 - A random variable with distribution P is observe din Gaussian noise and is estimated by a minimum meansquare estimator that assumes that the distribution is Q. This paper shows that the integral over all signal-to-noise ratios of the excess mean-square estimation error incurred by the mismatched estimator is twice the relative entropy D(P||Q). This representation of relative entropy can be generalized to non realvalued random variables, and can be particularized to give a new general representation of mutual information in terms of conditional means. Inspired by the new representation, we also propose a definition of free relative entropy which fills a gap in, and is consistent with, the literature on free probability.

AB - A random variable with distribution P is observe din Gaussian noise and is estimated by a minimum meansquare estimator that assumes that the distribution is Q. This paper shows that the integral over all signal-to-noise ratios of the excess mean-square estimation error incurred by the mismatched estimator is twice the relative entropy D(P||Q). This representation of relative entropy can be generalized to non realvalued random variables, and can be particularized to give a new general representation of mutual information in terms of conditional means. Inspired by the new representation, we also propose a definition of free relative entropy which fills a gap in, and is consistent with, the literature on free probability.

UR - http://www.scopus.com/inward/record.url?scp=70449473669&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70449473669&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2009.5205651

DO - 10.1109/ISIT.2009.5205651

M3 - Conference contribution

AN - SCOPUS:70449473669

SN - 9781424443130

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 809

EP - 813

BT - 2009 IEEE International Symposium on Information Theory, ISIT 2009

T2 - 2009 IEEE International Symposium on Information Theory, ISIT 2009

Y2 - 28 June 2009 through 3 July 2009

ER -