On Nonparametric Estimation of the Fisher Information

Wei Cao; Alex Dytso; Michael Faus; H. Vincent Poor; Gang Feng

doi:10.1109/ISIT44484.2020.9174450

On Nonparametric Estimation of the Fisher Information

Wei Cao, Alex Dytso, Michael Faus, H. Vincent Poor, Gang Feng

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Scopus citations

Abstract

This paper considers a problem of estimation of the Fisher information for location from a random sample of size n. First, an estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. Second, a new estimator, termed clipped estimator, is proposed. The new estimator is shown to have superior rates of convergence as compared to the Bhattacharya estimator, albeit with different regularity conditions. Third, both of the estimators are evaluated for the practically relevant case of a random variable contaminated by Gaussian noise. Moreover, using Brown's identity, which relates the Fisher information to the minimum mean squared error (MMSE) in Gaussian noise, a consistent estimator for the MMSE is proposed.

Original language	English (US)
Title of host publication	2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2216-2221
Number of pages	6
ISBN (Electronic)	9781728164328
DOIs	https://doi.org/10.1109/ISIT44484.2020.9174450
State	Published - Jun 2020
Externally published	Yes
Event	2020 IEEE International Symposium on Information Theory, ISIT 2020 - Los Angeles, United States Duration: Jul 21 2020 → Jul 26 2020

Publication series

Name	IEEE International Symposium on Information Theory - Proceedings
Volume	2020-June
ISSN (Print)	2157-8095

Conference

Conference	2020 IEEE International Symposium on Information Theory, ISIT 2020
Country/Territory	United States
City	Los Angeles
Period	7/21/20 → 7/26/20

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Information Systems
Modeling and Simulation
Applied Mathematics

Keywords

Fisher information
MMSE
Nonparametric estimation
kernel estimation

Access to Document

10.1109/ISIT44484.2020.9174450

Cite this

Cao, W., Dytso, A., Faus, M., Poor, H. V., & Feng, G. (2020). On Nonparametric Estimation of the Fisher Information. In 2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings (pp. 2216-2221). Article 9174450 (IEEE International Symposium on Information Theory - Proceedings; Vol. 2020-June). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT44484.2020.9174450

@inproceedings{81680670bbe4487e8fa41c82df10182c,

title = "On Nonparametric Estimation of the Fisher Information",

abstract = "This paper considers a problem of estimation of the Fisher information for location from a random sample of size n. First, an estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. Second, a new estimator, termed clipped estimator, is proposed. The new estimator is shown to have superior rates of convergence as compared to the Bhattacharya estimator, albeit with different regularity conditions. Third, both of the estimators are evaluated for the practically relevant case of a random variable contaminated by Gaussian noise. Moreover, using Brown's identity, which relates the Fisher information to the minimum mean squared error (MMSE) in Gaussian noise, a consistent estimator for the MMSE is proposed.",

keywords = "Fisher information, MMSE, Nonparametric estimation, kernel estimation",

author = "Wei Cao and Alex Dytso and Michael Faus and Poor, {H. Vincent} and Gang Feng",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 2020 IEEE International Symposium on Information Theory, ISIT 2020 ; Conference date: 21-07-2020 Through 26-07-2020",

year = "2020",

month = jun,

doi = "10.1109/ISIT44484.2020.9174450",

language = "English (US)",

series = "IEEE International Symposium on Information Theory - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2216--2221",

booktitle = "2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings",

address = "United States",

}

Cao, W, Dytso, A, Faus, M, Poor, HV & Feng, G 2020, On Nonparametric Estimation of the Fisher Information. in 2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings., 9174450, IEEE International Symposium on Information Theory - Proceedings, vol. 2020-June, Institute of Electrical and Electronics Engineers Inc., pp. 2216-2221, 2020 IEEE International Symposium on Information Theory, ISIT 2020, Los Angeles, United States, 7/21/20. https://doi.org/10.1109/ISIT44484.2020.9174450

On Nonparametric Estimation of the Fisher Information. / Cao, Wei; Dytso, Alex; Faus, Michael et al.
2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2020. p. 2216-2221 9174450 (IEEE International Symposium on Information Theory - Proceedings; Vol. 2020-June).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - On Nonparametric Estimation of the Fisher Information

AU - Cao, Wei

AU - Dytso, Alex

AU - Faus, Michael

AU - Poor, H. Vincent

AU - Feng, Gang

PY - 2020/6

Y1 - 2020/6

N2 - This paper considers a problem of estimation of the Fisher information for location from a random sample of size n. First, an estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. Second, a new estimator, termed clipped estimator, is proposed. The new estimator is shown to have superior rates of convergence as compared to the Bhattacharya estimator, albeit with different regularity conditions. Third, both of the estimators are evaluated for the practically relevant case of a random variable contaminated by Gaussian noise. Moreover, using Brown's identity, which relates the Fisher information to the minimum mean squared error (MMSE) in Gaussian noise, a consistent estimator for the MMSE is proposed.

AB - This paper considers a problem of estimation of the Fisher information for location from a random sample of size n. First, an estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. Second, a new estimator, termed clipped estimator, is proposed. The new estimator is shown to have superior rates of convergence as compared to the Bhattacharya estimator, albeit with different regularity conditions. Third, both of the estimators are evaluated for the practically relevant case of a random variable contaminated by Gaussian noise. Moreover, using Brown's identity, which relates the Fisher information to the minimum mean squared error (MMSE) in Gaussian noise, a consistent estimator for the MMSE is proposed.

KW - Fisher information

KW - MMSE

KW - Nonparametric estimation

KW - kernel estimation

UR - http://www.scopus.com/inward/record.url?scp=85090419576&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85090419576&partnerID=8YFLogxK

U2 - 10.1109/ISIT44484.2020.9174450

DO - 10.1109/ISIT44484.2020.9174450

M3 - Conference contribution

AN - SCOPUS:85090419576

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 2216

EP - 2221

BT - 2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2020 IEEE International Symposium on Information Theory, ISIT 2020

Y2 - 21 July 2020 through 26 July 2020

ER -

On Nonparametric Estimation of the Fisher Information

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this