### Abstract

We propose a provably optimal approximate dynamic programming algorithm for a class of multistage stochastic problems, taking into account that the probability distribution of the underlying stochastic process is not known and the state space is too large to be explored entirely. The algorithm and its proof of convergence rely on the fact that the optimal value functions of the problems within the problem class are concave and piecewise linear. The algorithm is a combination of Monte Carlo simulation, pure exploitation, stochastic approximation and a projection operation. Several applications, in areas like energy, control, inventory and finance, fall under the framework.

Original language | English (US) |
---|---|

Title of host publication | Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007 |

Pages | 52-59 |

Number of pages | 8 |

DOIs | |

State | Published - Sep 25 2007 |

Event | 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007 - Honolulu, HI, United States Duration: Apr 1 2007 → Apr 5 2007 |

### Publication series

Name | Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007 |
---|

### Other

Other | 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007 |
---|---|

Country | United States |

City | Honolulu, HI |

Period | 4/1/07 → 4/5/07 |

### All Science Journal Classification (ASJC) codes

- Computer Science Applications
- Software

## Fingerprint Dive into the research topics of 'An optimal ADP algorithm for a high-dimensional stochastic control problem'. Together they form a unique fingerprint.

## Cite this

*Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007*(pp. 52-59). [4220814] (Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007). https://doi.org/10.1109/ADPRL.2007.368169