Abstract
We review the literature on approximate dynamic programming, with the goal of better understanding the theory behind practical algorithms for solving dynamic programs with continuous and vector-valued states and actions and complex information processes. We build on the literature that has addressed the well-known problem of multidimensional (and possibly continuous) states, and the extensive literature on model-free dynamic programming, which also assumes that the expectation in Bellman's equation cannot be computed. However, we point out complications that arise when the actions/controls are vector-valued and possibly continuous. We then describe some recent research by the authors on approximate policy iteration algorithms that offer convergence guarantees (with technical assumptions) for both parametric and nonparametric architectures for the value function.
Original language | English (US) |
---|---|
Pages (from-to) | 336-352 |
Number of pages | 17 |
Journal | Journal of Control Theory and Applications |
Volume | 9 |
Issue number | 3 |
DOIs | |
State | Published - Aug 2011 |
All Science Journal Classification (ASJC) codes
- Control and Systems Engineering
- Hardware and Architecture
- Computer Science Applications
Keywords
- Approximate dynamic programming
- Approximation algorithms
- Optimal control
- Reinforcement learning