TY - GEN

T1 - An optimal ADP algorithm for a high-dimensional stochastic control problem

AU - Nascimento, Juliana

AU - Powell, Warren Buckler

N1 - Copyright:
Copyright 2008 Elsevier B.V., All rights reserved.

PY - 2007

Y1 - 2007

N2 - We propose a provably optimal approximate dynamic programming algorithm for a class of multistage stochastic problems, taking into account that the probability distribution of the underlying stochastic process is not known and the state space is too large to be explored entirely. The algorithm and its proof of convergence rely on the fact that the optimal value functions of the problems within the problem class are concave and piecewise linear. The algorithm is a combination of Monte Carlo simulation, pure exploitation, stochastic approximation and a projection operation. Several applications, in areas like energy, control, inventory and finance, fall under the framework.

AB - We propose a provably optimal approximate dynamic programming algorithm for a class of multistage stochastic problems, taking into account that the probability distribution of the underlying stochastic process is not known and the state space is too large to be explored entirely. The algorithm and its proof of convergence rely on the fact that the optimal value functions of the problems within the problem class are concave and piecewise linear. The algorithm is a combination of Monte Carlo simulation, pure exploitation, stochastic approximation and a projection operation. Several applications, in areas like energy, control, inventory and finance, fall under the framework.

UR - http://www.scopus.com/inward/record.url?scp=34548718915&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34548718915&partnerID=8YFLogxK

U2 - 10.1109/ADPRL.2007.368169

DO - 10.1109/ADPRL.2007.368169

M3 - Conference contribution

AN - SCOPUS:34548718915

SN - 1424407060

SN - 9781424407064

T3 - Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007

SP - 52

EP - 59

BT - Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007

T2 - 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007

Y2 - 1 April 2007 through 5 April 2007

ER -