Finite-Sample Risk Bounds for Maximum Likelihood Estimation with Arbitrary Penalties

W. D. Brinda, Jason M. Klusowski

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


The minimum description length two-part coding index of resolvability provides a finite-sample upper bound on the statistical risk of penalized likelihood estimators over countable models. However, the bound does not apply to unpenalized maximum likelihood estimation or procedures with exceedingly small penalties. In this paper, we point out a more general inequality that holds for arbitrary penalties. In addition, this approach makes it possible to derive exact risk bounds of order 1/n for iid parametric models, which improves on the order (log n)/n resolvability bounds. We conclude by discussing implications for adaptive estimation.

Original languageEnglish (US)
Pages (from-to)2727-2741
Number of pages15
JournalIEEE Transactions on Information Theory
Issue number4
StatePublished - Apr 2018
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences


  • Penalized likelihood estimation
  • codelength
  • minimum description length
  • redundancy
  • statistical risk


Dive into the research topics of 'Finite-Sample Risk Bounds for Maximum Likelihood Estimation with Arbitrary Penalties'. Together they form a unique fingerprint.

Cite this