TY - JOUR

T1 - Ergodicity, decisions, and partial information

AU - van Handel, Ramon

PY - 2014/1/1

Y1 - 2014/1/1

N2 - In the simplest sequential decision problem for an ergodic stochastic process X, at each time n a decision un is made as a function of past observations X0; : : : ;Xn_1, and a loss l.un;Xn/ is incurred. In this setting, it is known that one may choose (under a mild integrability assumption) a decision strategy whose pathwise time-average loss is asymptotically smaller than that of any other strategy. The corresponding problem in the case of partial information proves to be much more delicate, however: if the process X is not observable, but decisions must be based on the observation of a different process Y , the existence of pathwise optimal strategies is not guaranteed. The aim of this paper is to exhibit connections between pathwise optimal strategies and notions from ergodic theory. The sequential decision problem is developed in the general setting of an ergodic dynamical system .˝;B; P; T / with partial information Y _ B. The existence of pathwise optimal strategies grounded in two basic properties: the conditional ergodic theory of the dynamical system, and the complexity of the loss function. When the loss function is not too complex, a general sufficient condition for the existence of pathwise optimal strategies is that the dynamical system is a conditional K-automorphism relative to the past observationsWn_0 T nY. If the conditional ergodicity assumption is strengthened, the complexity assumption can be weakened. Several examples demonstrate the interplay between complexity and ergodicity, which does not arise in the case of full information. Our results also yield a decision-theoretic characterization of weak mixing in ergodic theory, and establish pathwise optimality of ergodic nonlinear filters.

AB - In the simplest sequential decision problem for an ergodic stochastic process X, at each time n a decision un is made as a function of past observations X0; : : : ;Xn_1, and a loss l.un;Xn/ is incurred. In this setting, it is known that one may choose (under a mild integrability assumption) a decision strategy whose pathwise time-average loss is asymptotically smaller than that of any other strategy. The corresponding problem in the case of partial information proves to be much more delicate, however: if the process X is not observable, but decisions must be based on the observation of a different process Y , the existence of pathwise optimal strategies is not guaranteed. The aim of this paper is to exhibit connections between pathwise optimal strategies and notions from ergodic theory. The sequential decision problem is developed in the general setting of an ergodic dynamical system .˝;B; P; T / with partial information Y _ B. The existence of pathwise optimal strategies grounded in two basic properties: the conditional ergodic theory of the dynamical system, and the complexity of the loss function. When the loss function is not too complex, a general sufficient condition for the existence of pathwise optimal strategies is that the dynamical system is a conditional K-automorphism relative to the past observationsWn_0 T nY. If the conditional ergodicity assumption is strengthened, the complexity assumption can be weakened. Several examples demonstrate the interplay between complexity and ergodicity, which does not arise in the case of full information. Our results also yield a decision-theoretic characterization of weak mixing in ergodic theory, and establish pathwise optimality of ergodic nonlinear filters.

UR - http://www.scopus.com/inward/record.url?scp=84927131880&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84927131880&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-11970-0_18

DO - 10.1007/978-3-319-11970-0_18

M3 - Article

AN - SCOPUS:84927131880

VL - 2123

SP - 411

EP - 459

JO - Lecture Notes in Mathematics

JF - Lecture Notes in Mathematics

SN - 0075-8434

ER -