MMSE dimension

Yihong Wu, Sergio Verdú

Research output: Contribution to journalArticle

43 Scopus citations

Abstract

If N is standard Gaussian, the minimum mean-square error (MMSE) of estimating a random variable X based on √snrX + N vanishes at least as fast as 1/snr as snr → ∞. We define the MMSE dimension of X as the limit as snr → ∞ of the product of snr and the MMSE. MMSE dimension is also shown to be the asymptotic ratio of nonlinear MMSE to linear MMSE. For discrete, absolutely continuous or mixed distribution we show that MMSE dimension equals Rnyi's information dimension. However, for a class of self-similar singular X (e.g., Cantor distribution), we show that the product of snr and MMSE oscillates around information dimension periodically in snr (dB). We also show that these results extend considerably beyond Gaussian noise under various technical conditions.

Original languageEnglish (US)
Article number5961853
Pages (from-to)4857-4879
Number of pages23
JournalIEEE Transactions on Information Theory
Volume57
Issue number8
DOIs
StatePublished - Aug 1 2011

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Keywords

  • Additive noise
  • Bayesian statistics
  • Gaussian noise
  • Rnyi information dimension
  • high-SNR asymptotics
  • minimum mean-square error (MMSE)
  • mutual information
  • non-Gaussian noise

Fingerprint Dive into the research topics of 'MMSE dimension'. Together they form a unique fingerprint.

  • Cite this

    Wu, Y., & Verdú, S. (2011). MMSE dimension. IEEE Transactions on Information Theory, 57(8), 4857-4879. [5961853]. https://doi.org/10.1109/TIT.2011.2158905