An optimal ADP algorithm for a high-dimensional stochastic control problem

Juliana Nascimento, Warren Buckler Powell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

We propose a provably optimal approximate dynamic programming algorithm for a class of multistage stochastic problems, taking into account that the probability distribution of the underlying stochastic process is not known and the state space is too large to be explored entirely. The algorithm and its proof of convergence rely on the fact that the optimal value functions of the problems within the problem class are concave and piecewise linear. The algorithm is a combination of Monte Carlo simulation, pure exploitation, stochastic approximation and a projection operation. Several applications, in areas like energy, control, inventory and finance, fall under the framework.

Original languageEnglish (US)
Title of host publicationProceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007
Pages52-59
Number of pages8
DOIs
StatePublished - Sep 25 2007
Event2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007 - Honolulu, HI, United States
Duration: Apr 1 2007Apr 5 2007

Publication series

NameProceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007

Other

Other2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007
CountryUnited States
CityHonolulu, HI
Period4/1/074/5/07

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Software

Fingerprint Dive into the research topics of 'An optimal ADP algorithm for a high-dimensional stochastic control problem'. Together they form a unique fingerprint.

  • Cite this

    Nascimento, J., & Powell, W. B. (2007). An optimal ADP algorithm for a high-dimensional stochastic control problem. In Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007 (pp. 52-59). [4220814] (Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007). https://doi.org/10.1109/ADPRL.2007.368169