An optimal ADP algorithm for a high-dimensional stochastic control problem

Juliana Nascimento; Warren Buckler Powell

doi:10.1109/ADPRL.2007.368169

An optimal ADP algorithm for a high-dimensional stochastic control problem

Juliana Nascimento, Warren Buckler Powell

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Scopus citations

Abstract

We propose a provably optimal approximate dynamic programming algorithm for a class of multistage stochastic problems, taking into account that the probability distribution of the underlying stochastic process is not known and the state space is too large to be explored entirely. The algorithm and its proof of convergence rely on the fact that the optimal value functions of the problems within the problem class are concave and piecewise linear. The algorithm is a combination of Monte Carlo simulation, pure exploitation, stochastic approximation and a projection operation. Several applications, in areas like energy, control, inventory and finance, fall under the framework.

Original language	English (US)
Title of host publication	Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007
Pages	52-59
Number of pages	8
DOIs	https://doi.org/10.1109/ADPRL.2007.368169
State	Published - 2007
Event	2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007 - Honolulu, HI, United States Duration: Apr 1 2007 → Apr 5 2007

Publication series

Name	Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007

Other

Other	2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007
Country/Territory	United States
City	Honolulu, HI
Period	4/1/07 → 4/5/07

All Science Journal Classification (ASJC) codes

Computer Science Applications
Software

Access to Document

10.1109/ADPRL.2007.368169

Cite this

Nascimento, J., & Powell, W. B. (2007). An optimal ADP algorithm for a high-dimensional stochastic control problem. In Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007 (pp. 52-59). Article 4220814 (Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007). https://doi.org/10.1109/ADPRL.2007.368169

@inproceedings{9d86af6f485e414488b1797220019a72,

title = "An optimal ADP algorithm for a high-dimensional stochastic control problem",

abstract = "We propose a provably optimal approximate dynamic programming algorithm for a class of multistage stochastic problems, taking into account that the probability distribution of the underlying stochastic process is not known and the state space is too large to be explored entirely. The algorithm and its proof of convergence rely on the fact that the optimal value functions of the problems within the problem class are concave and piecewise linear. The algorithm is a combination of Monte Carlo simulation, pure exploitation, stochastic approximation and a projection operation. Several applications, in areas like energy, control, inventory and finance, fall under the framework.",

author = "Juliana Nascimento and Powell, {Warren Buckler}",

year = "2007",

doi = "10.1109/ADPRL.2007.368169",

language = "English (US)",

isbn = "1424407060",

series = "Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007",

pages = "52--59",

booktitle = "Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007",

note = "2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007 ; Conference date: 01-04-2007 Through 05-04-2007",

}

Nascimento, J & Powell, WB 2007, An optimal ADP algorithm for a high-dimensional stochastic control problem. in Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007., 4220814, Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007, pp. 52-59, 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007, Honolulu, HI, United States, 4/1/07. https://doi.org/10.1109/ADPRL.2007.368169

An optimal ADP algorithm for a high-dimensional stochastic control problem. / Nascimento, Juliana; Powell, Warren Buckler.
Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007. 2007. p. 52-59 4220814 (Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - An optimal ADP algorithm for a high-dimensional stochastic control problem

AU - Nascimento, Juliana

AU - Powell, Warren Buckler

PY - 2007

Y1 - 2007

N2 - We propose a provably optimal approximate dynamic programming algorithm for a class of multistage stochastic problems, taking into account that the probability distribution of the underlying stochastic process is not known and the state space is too large to be explored entirely. The algorithm and its proof of convergence rely on the fact that the optimal value functions of the problems within the problem class are concave and piecewise linear. The algorithm is a combination of Monte Carlo simulation, pure exploitation, stochastic approximation and a projection operation. Several applications, in areas like energy, control, inventory and finance, fall under the framework.

AB - We propose a provably optimal approximate dynamic programming algorithm for a class of multistage stochastic problems, taking into account that the probability distribution of the underlying stochastic process is not known and the state space is too large to be explored entirely. The algorithm and its proof of convergence rely on the fact that the optimal value functions of the problems within the problem class are concave and piecewise linear. The algorithm is a combination of Monte Carlo simulation, pure exploitation, stochastic approximation and a projection operation. Several applications, in areas like energy, control, inventory and finance, fall under the framework.

UR - http://www.scopus.com/inward/record.url?scp=34548718915&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34548718915&partnerID=8YFLogxK

U2 - 10.1109/ADPRL.2007.368169

DO - 10.1109/ADPRL.2007.368169

M3 - Conference contribution

AN - SCOPUS:34548718915

SN - 1424407060

SN - 9781424407064

T3 - Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007

SP - 52

EP - 59

BT - Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007

T2 - 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007

Y2 - 1 April 2007 through 5 April 2007

ER -

Nascimento J, Powell WB. An optimal ADP algorithm for a high-dimensional stochastic control problem. In Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007. 2007. p. 52-59. 4220814. (Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007). doi: 10.1109/ADPRL.2007.368169

An optimal ADP algorithm for a high-dimensional stochastic control problem

Abstract

Publication series

Other

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this