Bayesian active learning with basis functions

Ilya O. Ryzhov; Warren Buckler Powell

doi:10.1109/ADPRL.2011.5967365

Bayesian active learning with basis functions

Ilya O. Ryzhov, Warren Buckler Powell

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

8 Scopus citations

Abstract

A common technique for dealing with the curse of dimensionality in approximate dynamic programming is to use a parametric value function approximation, where the value of being in a state is assumed to be a linear combination of basis functions. Even with this simplification, we face the exploration/exploitation dilemma: an inaccurate approximation may lead to poor decisions, making it necessary to sometimes explore actions that appear to be suboptimal. We propose a Bayesian strategy for active learning with basis functions, based on the knowledge gradient concept from the optimal learning literature. The new method performs well in numerical experiments conducted on an energy storage problem.

Original language	English (US)
Title of host publication	IEEE SSCI 2011
Subtitle of host publication	Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning
Pages	143-150
Number of pages	8
DOIs	https://doi.org/10.1109/ADPRL.2011.5967365
State	Published - Sep 5 2011
Event	Symposium Series on Computational Intelligence, IEEE SSCI2011 - 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2011 - Paris, France Duration: Apr 11 2011 → Apr 15 2011

Publication series

Name	IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning

Other

Other	Symposium Series on Computational Intelligence, IEEE SSCI2011 - 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2011
Country/Territory	France
City	Paris
Period	4/11/11 → 4/15/11

All Science Journal Classification (ASJC) codes

Computer Science Applications
Software

Access to Document

10.1109/ADPRL.2011.5967365

Cite this

Ryzhov, I. O., & Powell, W. B. (2011). Bayesian active learning with basis functions. In IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (pp. 143-150). Article 5967365 (IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning). https://doi.org/10.1109/ADPRL.2011.5967365

Ryzhov, Ilya O. ; Powell, Warren Buckler. / Bayesian active learning with basis functions. IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning. 2011. pp. 143-150 (IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning).

@inproceedings{1f4a78505e5c42548f95764967b905db,

title = "Bayesian active learning with basis functions",

abstract = "A common technique for dealing with the curse of dimensionality in approximate dynamic programming is to use a parametric value function approximation, where the value of being in a state is assumed to be a linear combination of basis functions. Even with this simplification, we face the exploration/exploitation dilemma: an inaccurate approximation may lead to poor decisions, making it necessary to sometimes explore actions that appear to be suboptimal. We propose a Bayesian strategy for active learning with basis functions, based on the knowledge gradient concept from the optimal learning literature. The new method performs well in numerical experiments conducted on an energy storage problem.",

author = "Ryzhov, {Ilya O.} and Powell, {Warren Buckler}",

year = "2011",

month = sep,

day = "5",

doi = "10.1109/ADPRL.2011.5967365",

language = "English (US)",

isbn = "9781424498888",

series = "IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning",

pages = "143--150",

booktitle = "IEEE SSCI 2011",

note = "Symposium Series on Computational Intelligence, IEEE SSCI2011 - 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2011 ; Conference date: 11-04-2011 Through 15-04-2011",

}

Ryzhov, IO & Powell, WB 2011, Bayesian active learning with basis functions. in IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning., 5967365, IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, pp. 143-150, Symposium Series on Computational Intelligence, IEEE SSCI2011 - 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2011, Paris, France, 4/11/11. https://doi.org/10.1109/ADPRL.2011.5967365

Bayesian active learning with basis functions. / Ryzhov, Ilya O.; Powell, Warren Buckler.
IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning. 2011. p. 143-150 5967365 (IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Bayesian active learning with basis functions

AU - Ryzhov, Ilya O.

AU - Powell, Warren Buckler

PY - 2011/9/5

Y1 - 2011/9/5

N2 - A common technique for dealing with the curse of dimensionality in approximate dynamic programming is to use a parametric value function approximation, where the value of being in a state is assumed to be a linear combination of basis functions. Even with this simplification, we face the exploration/exploitation dilemma: an inaccurate approximation may lead to poor decisions, making it necessary to sometimes explore actions that appear to be suboptimal. We propose a Bayesian strategy for active learning with basis functions, based on the knowledge gradient concept from the optimal learning literature. The new method performs well in numerical experiments conducted on an energy storage problem.

AB - A common technique for dealing with the curse of dimensionality in approximate dynamic programming is to use a parametric value function approximation, where the value of being in a state is assumed to be a linear combination of basis functions. Even with this simplification, we face the exploration/exploitation dilemma: an inaccurate approximation may lead to poor decisions, making it necessary to sometimes explore actions that appear to be suboptimal. We propose a Bayesian strategy for active learning with basis functions, based on the knowledge gradient concept from the optimal learning literature. The new method performs well in numerical experiments conducted on an energy storage problem.

UR - http://www.scopus.com/inward/record.url?scp=80052219755&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80052219755&partnerID=8YFLogxK

U2 - 10.1109/ADPRL.2011.5967365

DO - 10.1109/ADPRL.2011.5967365

M3 - Conference contribution

AN - SCOPUS:80052219755

SN - 9781424498888

T3 - IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning

SP - 143

EP - 150

BT - IEEE SSCI 2011

T2 - Symposium Series on Computational Intelligence, IEEE SSCI2011 - 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2011

Y2 - 11 April 2011 through 15 April 2011

ER -

Ryzhov IO, Powell WB. Bayesian active learning with basis functions. In IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning. 2011. p. 143-150. 5967365. (IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning). doi: 10.1109/ADPRL.2011.5967365

Bayesian active learning with basis functions

Abstract

Publication series

Other

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this