Bayesian Risk with Bregman Loss: A Cramér-Rao Type Bound and Linear Estimation

Alex Dytso; Michael Faub; H. Vincent Poor

doi:10.1109/TIT.2021.3130381

Bayesian Risk with Bregman Loss: A Cramér-Rao Type Bound and Linear Estimation

Alex Dytso, Michael Faub, H. Vincent Poor

Research output: Contribution to journal › Article › peer-review

5 Scopus citations

Abstract

A general class of Bayesian lower bounds when the underlying loss function is a Bregman divergence is demonstrated. This class can be considered as an extension of the Weinstein-Weiss family of bounds for the mean squared error and relies on finding a variational characterization of Bayesian risk. This approach allows for the derivation of a version of the Cramér-Rao bound that is specific to a given Bregman divergence. This new generalization of the Cramér-Rao bound reduces to the classical one when the loss function is taken to be the Euclidean norm. In order to evaluate the effectiveness of the new lower bounds, the paper also develops upper bounds on Bayesian risk, which are based on optimal linear estimators. The effectiveness of the new bound is evaluated in the Poisson noise setting.

Original language	English (US)
Pages (from-to)	1985-2000
Number of pages	16
Journal	IEEE Transactions on Information Theory
Volume	68
Issue number	3
DOIs	https://doi.org/10.1109/TIT.2021.3130381
State	Published - Mar 1 2022
Externally published	Yes

All Science Journal Classification (ASJC) codes

Information Systems
Computer Science Applications
Library and Information Sciences

Keywords

Bregman divergence
Cramér-Rao
Gaussian Noise
Linear estimation
Poisson noise
minimum mean squared error (MMSE)

Access to Document

10.1109/TIT.2021.3130381

Cite this

@article{e8d6573f3993498eb97c5bb17b8a4025,

title = "Bayesian Risk with Bregman Loss: A Cram{\'e}r-Rao Type Bound and Linear Estimation",

abstract = "A general class of Bayesian lower bounds when the underlying loss function is a Bregman divergence is demonstrated. This class can be considered as an extension of the Weinstein-Weiss family of bounds for the mean squared error and relies on finding a variational characterization of Bayesian risk. This approach allows for the derivation of a version of the Cram{\'e}r-Rao bound that is specific to a given Bregman divergence. This new generalization of the Cram{\'e}r-Rao bound reduces to the classical one when the loss function is taken to be the Euclidean norm. In order to evaluate the effectiveness of the new lower bounds, the paper also develops upper bounds on Bayesian risk, which are based on optimal linear estimators. The effectiveness of the new bound is evaluated in the Poisson noise setting.",

keywords = "Bregman divergence, Cram{\'e}r-Rao, Gaussian Noise, Linear estimation, Poisson noise, minimum mean squared error (MMSE)",

author = "Alex Dytso and Michael Faub and {Vincent Poor}, H.",

note = "Publisher Copyright: {\textcopyright} 1963-2012 IEEE.",

year = "2022",

month = mar,

day = "1",

doi = "10.1109/TIT.2021.3130381",

language = "English (US)",

volume = "68",

pages = "1985--2000",

journal = "IEEE Transactions on Information Theory",

issn = "0018-9448",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "3",

}

TY - JOUR

T1 - Bayesian Risk with Bregman Loss

T2 - A Cramér-Rao Type Bound and Linear Estimation

AU - Dytso, Alex

AU - Faub, Michael

AU - Vincent Poor, H.

PY - 2022/3/1

Y1 - 2022/3/1

N2 - A general class of Bayesian lower bounds when the underlying loss function is a Bregman divergence is demonstrated. This class can be considered as an extension of the Weinstein-Weiss family of bounds for the mean squared error and relies on finding a variational characterization of Bayesian risk. This approach allows for the derivation of a version of the Cramér-Rao bound that is specific to a given Bregman divergence. This new generalization of the Cramér-Rao bound reduces to the classical one when the loss function is taken to be the Euclidean norm. In order to evaluate the effectiveness of the new lower bounds, the paper also develops upper bounds on Bayesian risk, which are based on optimal linear estimators. The effectiveness of the new bound is evaluated in the Poisson noise setting.

AB - A general class of Bayesian lower bounds when the underlying loss function is a Bregman divergence is demonstrated. This class can be considered as an extension of the Weinstein-Weiss family of bounds for the mean squared error and relies on finding a variational characterization of Bayesian risk. This approach allows for the derivation of a version of the Cramér-Rao bound that is specific to a given Bregman divergence. This new generalization of the Cramér-Rao bound reduces to the classical one when the loss function is taken to be the Euclidean norm. In order to evaluate the effectiveness of the new lower bounds, the paper also develops upper bounds on Bayesian risk, which are based on optimal linear estimators. The effectiveness of the new bound is evaluated in the Poisson noise setting.

KW - Bregman divergence

KW - Cramér-Rao

KW - Gaussian Noise

KW - Linear estimation

KW - Poisson noise

KW - minimum mean squared error (MMSE)

UR - http://www.scopus.com/inward/record.url?scp=85120070124&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85120070124&partnerID=8YFLogxK

U2 - 10.1109/TIT.2021.3130381

DO - 10.1109/TIT.2021.3130381

M3 - Article

AN - SCOPUS:85120070124

SN - 0018-9448

VL - 68

SP - 1985

EP - 2000

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 3

ER -

Bayesian Risk with Bregman Loss: A Cramér-Rao Type Bound and Linear Estimation

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this