Regularization of wavelet approximations

Anestis Antoniadis, Jianqing Fan

Research output: Contribution to journalArticlepeer-review

391 Scopus citations


In this paper, we introduce nonlinear regularized wavelet estimators for estimating nonparametric regression functions when sampling points are not uniformly spaced. The approach can apply readily to many other statistical contexts. Various new penalty functions are proposed. The hard-thresholding and soft-thresholding estimators of Donoho and Johnstone are specific members of nonlinear regularized wavelet estimators. They correspond to the lower and upper envelopes of a class of the penalized least squares estimators. Necessary conditions for penalty functions are given for regularized estimators to possess thresholding properties. Oracle inequalities and universal thresholding parameters are obtained for a large class of penalty functions. The sampling properties of nonlinear regularized wavelet estimators are established and are shown to be adaptively minimax. To efficiently solve penalized least squares problems, nonlinear regularized Sobolev interpolators (NRSI) are proposed as initial estimators, which are shown to have good sampling properties. The NRSI is further ameliorated by regularized one-step estimators, which are the one-step estimators of the penalized least squares problems using the NRSI as initial estimators. The graduated nonconvexity algorithm is also introduced to handle penalized least squares problems. The newly introduced approaches are illustrated by a few numerical examples.

Original languageEnglish (US)
Pages (from-to)939-955
Number of pages17
JournalJournal of the American Statistical Association
Issue number455
StatePublished - Sep 1 2001
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty


  • Asymptotic minimax
  • Irregular designs
  • Nonquadratic penality functions
  • Oracle inequalities
  • Penalized least-squares
  • ROSE
  • Wavelets


Dive into the research topics of 'Regularization of wavelet approximations'. Together they form a unique fingerprint.

Cite this