The value of information in multi-armed bandits with exponentially distributed rewards

Ilya O. Ryzhov; Warren Buckler Powell

doi:10.1016/j.procs.2011.04.147

The value of information in multi-armed bandits with exponentially distributed rewards

Ilya O. Ryzhov, Warren Buckler Powell

Research output: Contribution to journal › Conference article › peer-review

7 Scopus citations

Abstract

We consider a class of multi-armed bandit problems where the reward obtained by pulling an arm is drawn from an exponential distribution whose parameter is unknown. A Bayesian model with independent gamma priors is used to represent our beliefs and uncertainty about the exponential parameters. We derive a precise expression for the marginal value of information in this problem, which allows us to create a new knowledge gradient (KG) policy for making decisions. The policy is practical and easy to implement, making a case for value of information as a general approach to optimal learning problems with many different types of learning models.

Original language	English (US)
Pages (from-to)	1363-1372
Number of pages	10
Journal	Procedia Computer Science
Volume	4
DOIs	https://doi.org/10.1016/j.procs.2011.04.147
State	Published - 2011
Event	11th International Conference on Computational Science, ICCS 2011 - Singapore, Singapore Duration: Jun 1 2011 → Jun 3 2011

All Science Journal Classification (ASJC) codes

General Computer Science

Keywords

Exponential rewards
Knowledge gradient
Multi-armed bandit
Optimal learning

Access to Document

10.1016/j.procs.2011.04.147

Cite this

@article{2ba2fe2d9d3b4203a79d8108129e782b,

title = "The value of information in multi-armed bandits with exponentially distributed rewards",

abstract = "We consider a class of multi-armed bandit problems where the reward obtained by pulling an arm is drawn from an exponential distribution whose parameter is unknown. A Bayesian model with independent gamma priors is used to represent our beliefs and uncertainty about the exponential parameters. We derive a precise expression for the marginal value of information in this problem, which allows us to create a new knowledge gradient (KG) policy for making decisions. The policy is practical and easy to implement, making a case for value of information as a general approach to optimal learning problems with many different types of learning models.",

keywords = "Exponential rewards, Knowledge gradient, Multi-armed bandit, Optimal learning",

author = "Ryzhov, {Ilya O.} and Powell, {Warren Buckler}",

note = "Funding Information: This work was supported in part by AFOSR contract FA9550-08-1-0195 through the Center for Dynamic Data Analysis.; 11th International Conference on Computational Science, ICCS 2011 ; Conference date: 01-06-2011 Through 03-06-2011",

year = "2011",

doi = "10.1016/j.procs.2011.04.147",

language = "English (US)",

volume = "4",

pages = "1363--1372",

journal = "Procedia Computer Science",

issn = "1877-0509",

publisher = "Elsevier BV",

}

TY - JOUR

T1 - The value of information in multi-armed bandits with exponentially distributed rewards

AU - Ryzhov, Ilya O.

AU - Powell, Warren Buckler

N1 - Funding Information: This work was supported in part by AFOSR contract FA9550-08-1-0195 through the Center for Dynamic Data Analysis.

PY - 2011

Y1 - 2011

N2 - We consider a class of multi-armed bandit problems where the reward obtained by pulling an arm is drawn from an exponential distribution whose parameter is unknown. A Bayesian model with independent gamma priors is used to represent our beliefs and uncertainty about the exponential parameters. We derive a precise expression for the marginal value of information in this problem, which allows us to create a new knowledge gradient (KG) policy for making decisions. The policy is practical and easy to implement, making a case for value of information as a general approach to optimal learning problems with many different types of learning models.

AB - We consider a class of multi-armed bandit problems where the reward obtained by pulling an arm is drawn from an exponential distribution whose parameter is unknown. A Bayesian model with independent gamma priors is used to represent our beliefs and uncertainty about the exponential parameters. We derive a precise expression for the marginal value of information in this problem, which allows us to create a new knowledge gradient (KG) policy for making decisions. The policy is practical and easy to implement, making a case for value of information as a general approach to optimal learning problems with many different types of learning models.

KW - Exponential rewards

KW - Knowledge gradient

KW - Multi-armed bandit

KW - Optimal learning

UR - http://www.scopus.com/inward/record.url?scp=79958262308&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79958262308&partnerID=8YFLogxK

U2 - 10.1016/j.procs.2011.04.147

DO - 10.1016/j.procs.2011.04.147

M3 - Conference article

AN - SCOPUS:79958262308

SN - 1877-0509

VL - 4

SP - 1363

EP - 1372

JO - Procedia Computer Science

JF - Procedia Computer Science

T2 - 11th International Conference on Computational Science, ICCS 2011

Y2 - 1 June 2011 through 3 June 2011

ER -

The value of information in multi-armed bandits with exponentially distributed rewards

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this