TY - GEN

T1 - Optimal learning of transition probabilities in the two-agent newsvendor problem

AU - Ryzhov, Ilya O.

AU - Valdez-Vivas, Martin R.

AU - Powell, Warren Buckler

PY - 2010

Y1 - 2010

N2 - We examine a newsvendor problem with two agents: a requesting agent that observes private demand information, and an oversight agent that must determine how to allocate resources upon receiving a bid from the requesting agent. Because the two agents have different cost structures, the requesting agent tends to bid higher than the amount that is actually needed. As a result, the allocating agent needs to adaptively learn how to interpret the bids and estimate the requesting agent's biases. Learning must occur as quickly as possible, because each suboptimal resource allocation incurs an economic cost. We present a mathematical model that casts the problem as a Markov decision process with unknown transition probabilities. We then perform a simulation study comparing four different techniques for optimal learning of transition probabilities. The best technique is shown to be a knowledge gradient algorithm, based on a one-period look-ahead approach.

AB - We examine a newsvendor problem with two agents: a requesting agent that observes private demand information, and an oversight agent that must determine how to allocate resources upon receiving a bid from the requesting agent. Because the two agents have different cost structures, the requesting agent tends to bid higher than the amount that is actually needed. As a result, the allocating agent needs to adaptively learn how to interpret the bids and estimate the requesting agent's biases. Learning must occur as quickly as possible, because each suboptimal resource allocation incurs an economic cost. We present a mathematical model that casts the problem as a Markov decision process with unknown transition probabilities. We then perform a simulation study comparing four different techniques for optimal learning of transition probabilities. The best technique is shown to be a knowledge gradient algorithm, based on a one-period look-ahead approach.

UR - http://www.scopus.com/inward/record.url?scp=79951586758&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79951586758&partnerID=8YFLogxK

U2 - 10.1109/WSC.2010.5679081

DO - 10.1109/WSC.2010.5679081

M3 - Conference contribution

AN - SCOPUS:79951586758

SN - 9781424498666

T3 - Proceedings - Winter Simulation Conference

SP - 1088

EP - 1098

BT - Proceedings of the 2010 Winter Simulation Conference, WSC'10

T2 - 2010 43rd Winter Simulation Conference, WSC'10

Y2 - 5 December 2010 through 8 December 2010

ER -