TY - GEN
T1 - Approximate dynamic programming with correlated Bayesian beliefs
AU - Ryzhov, Ilya O.
AU - Powell, Warren Buckler
N1 - Copyright:
Copyright 2011 Elsevier B.V., All rights reserved.
PY - 2010
Y1 - 2010
N2 - In approximate dynamic programming, we can represent our uncertainty about the value function using a Bayesian model with correlated beliefs. Thus, a decision made at a single state can provide us with information about many states, making each individual observation much more powerful. We propose a new exploration strategy based on the knowledge gradient concept from the optimal learning literature, which is currently the only method capable of handling correlated belief structures. The proposed method outperforms several other heuristics in numerical experiments conducted on two broad problem classes.
AB - In approximate dynamic programming, we can represent our uncertainty about the value function using a Bayesian model with correlated beliefs. Thus, a decision made at a single state can provide us with information about many states, making each individual observation much more powerful. We propose a new exploration strategy based on the knowledge gradient concept from the optimal learning literature, which is currently the only method capable of handling correlated belief structures. The proposed method outperforms several other heuristics in numerical experiments conducted on two broad problem classes.
UR - http://www.scopus.com/inward/record.url?scp=79952383607&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=79952383607&partnerID=8YFLogxK
U2 - 10.1109/ALLERTON.2010.5707072
DO - 10.1109/ALLERTON.2010.5707072
M3 - Conference contribution
AN - SCOPUS:79952383607
SN - 9781424482146
T3 - 2010 48th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2010
SP - 1360
EP - 1367
BT - 2010 48th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2010
T2 - 48th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2010
Y2 - 29 September 2010 through 1 October 2010
ER -