An optimal approximate dynamic programming algorithm for concave, scalar storage problems with vector-valued controls

Juliana Nascimento, Warren Buckler Powell

Research output: Contribution to journalArticle

43 Scopus citations

Abstract

We prove convergence of an approximate dynamic programming algorithm for a class of high-dimensional stochastic control problems linked by a scalar storage device, given a technical condition. Our problem is motivated by the problem of optimizing energy flows for a power grid supported by grid-level storage. The problem is formulated as a stochastic, dynamic program, where we estimate the value of resources in storage using a piecewise linear value function approximation. Given the technical condition, we provide a rigorous convergence proof for an approximate dynamic programming algorithm, which can capture the presence of both the amount of energy held in storage as well as other exogenous variables. Our algorithm exploits the natural concavity of the problem to avoid any need for explicit exploration policies.

Original languageEnglish (US)
Article number6557011
Pages (from-to)2995-3010
Number of pages16
JournalIEEE Transactions on Automatic Control
Volume58
Issue number12
DOIs
StatePublished - Dec 1 2013

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Computer Science Applications
  • Electrical and Electronic Engineering

Keywords

  • Approximate dynamic programming
  • Resource allocation
  • Storage

Fingerprint Dive into the research topics of 'An optimal approximate dynamic programming algorithm for concave, scalar storage problems with vector-valued controls'. Together they form a unique fingerprint.

  • Cite this