A comparison of approximate dynamic programming techniques on benchmark energy storage problems: Does anything work?

Daniel R. Jiang, Thuy V. Pham, Warren Buckler Powell, Daniel F. Salas, Warren R. Scott

Research output: Chapter in Book/Report/Conference proceedingConference contribution

21 Scopus citations

Abstract

As more renewable, yet volatile, forms of energy like solar and wind are being incorporated into the grid, the problem of finding optimal control policies for energy storage is becoming increasingly important. These sequential decision problems are often modeled as stochastic dynamic programs, but when the state space becomes large, traditional (exact) techniques such as backward induction, policy iteration, or value iteration quickly become computationally intractable. Approximate dynamic programming (ADP) thus becomes a natural solution technique for solving these problems to near-optimality using significantly fewer computational resources. In this paper, we compare the performance of the following: various approximation architectures with approximate policy iteration (API), approximate value iteration (AVI) with structured lookup table, and direct policy search on a benchmarked energy storage problem (i.e., the optimal solution is computable).

Original languageEnglish (US)
Title of host publicationIEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - ADPRL 2014
Subtitle of host publication2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781479945535
DOIs
StatePublished - Jan 14 2014
Event2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2014 - Orlando, United States
Duration: Dec 9 2014Dec 12 2014

Publication series

NameIEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - ADPRL 2014: 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, Proceedings

Other

Other2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2014
CountryUnited States
CityOrlando
Period12/9/1412/12/14

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Software

Fingerprint Dive into the research topics of 'A comparison of approximate dynamic programming techniques on benchmark energy storage problems: Does anything work?'. Together they form a unique fingerprint.

  • Cite this

    Jiang, D. R., Pham, T. V., Powell, W. B., Salas, D. F., & Scott, W. R. (2014). A comparison of approximate dynamic programming techniques on benchmark energy storage problems: Does anything work? In IEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - ADPRL 2014: 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, Proceedings [7010626] (IEEE SSCI 2014 - 2014 IEEE Symposium Series on Computational Intelligence - ADPRL 2014: 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ADPRL.2014.7010626