Approximate dynamic programming with correlated Bayesian beliefs

Ilya O. Ryzhov, Warren Buckler Powell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations

Abstract

In approximate dynamic programming, we can represent our uncertainty about the value function using a Bayesian model with correlated beliefs. Thus, a decision made at a single state can provide us with information about many states, making each individual observation much more powerful. We propose a new exploration strategy based on the knowledge gradient concept from the optimal learning literature, which is currently the only method capable of handling correlated belief structures. The proposed method outperforms several other heuristics in numerical experiments conducted on two broad problem classes.

Original languageEnglish (US)
Title of host publication2010 48th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2010
Pages1360-1367
Number of pages8
DOIs
StatePublished - Dec 1 2010
Event48th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2010 - Monticello, IL, United States
Duration: Sep 29 2010Oct 1 2010

Publication series

Name2010 48th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2010

Other

Other48th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2010
CountryUnited States
CityMonticello, IL
Period9/29/1010/1/10

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Control and Systems Engineering

Fingerprint Dive into the research topics of 'Approximate dynamic programming with correlated Bayesian beliefs'. Together they form a unique fingerprint.

  • Cite this

    Ryzhov, I. O., & Powell, W. B. (2010). Approximate dynamic programming with correlated Bayesian beliefs. In 2010 48th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2010 (pp. 1360-1367). [5707072] (2010 48th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2010). https://doi.org/10.1109/ALLERTON.2010.5707072