TY - CHAP

T1 - Complexity of model classes and smoothing noisy data

AU - Bartlett, Peter L.

AU - Kulkarni, Sanjeev R.

N1 - Copyright:
Copyright 2012 Elsevier B.V., All rights reserved.

PY - 1996

Y1 - 1996

N2 - We consider the problem of smoothing a sequence of noisy observations using a fixed class of models. Via a deterministic analysis, we obtain necessary and sufficient conditions on the noise sequence and model class that ensure that a class of natural estimators gives near optimal smoothing. In the case of i.i.d. random noise, we show that the accuracy of these estimators depends on a measure of complexity of the model class involving covering numbers. Our formulation and results are quite general and are related to a number of problems in learning, prediction, and estimation. As a special case, we consider an application to output smoothing for certain classes of linear and nonlinear systems. The performance of output smoothing is given in terms of natural complexity parameters of the model class, such as bounds on the order of linear systems, the l 1-norm of the impulse response of stable linear systems, or the memory of a Lipschitz nonlinear system satisfying a fading memory condition.

AB - We consider the problem of smoothing a sequence of noisy observations using a fixed class of models. Via a deterministic analysis, we obtain necessary and sufficient conditions on the noise sequence and model class that ensure that a class of natural estimators gives near optimal smoothing. In the case of i.i.d. random noise, we show that the accuracy of these estimators depends on a measure of complexity of the model class involving covering numbers. Our formulation and results are quite general and are related to a number of problems in learning, prediction, and estimation. As a special case, we consider an application to output smoothing for certain classes of linear and nonlinear systems. The performance of output smoothing is given in terms of natural complexity parameters of the model class, such as bounds on the order of linear systems, the l 1-norm of the impulse response of stable linear systems, or the memory of a Lipschitz nonlinear system satisfying a fading memory condition.

UR - http://www.scopus.com/inward/record.url?scp=0030386454&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0030386454&partnerID=8YFLogxK

M3 - Chapter

AN - SCOPUS:0030386454

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 2312

EP - 2317

BT - Proceedings of the IEEE Conference on Decision and Control

A2 - Anon, null

T2 - Proceedings of the 35th IEEE Conference on Decision and Control. Part 4 (of 4)

Y2 - 11 December 1996 through 13 December 1996

ER -