Optimal learning of transition probabilities in the two-agent newsvendor problem

Ilya O. Ryzhov, Martin R. Valdez-Vivas, Warren Buckler Powell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations


We examine a newsvendor problem with two agents: a requesting agent that observes private demand information, and an oversight agent that must determine how to allocate resources upon receiving a bid from the requesting agent. Because the two agents have different cost structures, the requesting agent tends to bid higher than the amount that is actually needed. As a result, the allocating agent needs to adaptively learn how to interpret the bids and estimate the requesting agent's biases. Learning must occur as quickly as possible, because each suboptimal resource allocation incurs an economic cost. We present a mathematical model that casts the problem as a Markov decision process with unknown transition probabilities. We then perform a simulation study comparing four different techniques for optimal learning of transition probabilities. The best technique is shown to be a knowledge gradient algorithm, based on a one-period look-ahead approach.

Original languageEnglish (US)
Title of host publicationProceedings of the 2010 Winter Simulation Conference, WSC'10
Number of pages11
StatePublished - 2010
Event2010 43rd Winter Simulation Conference, WSC'10 - Baltimore, MD, United States
Duration: Dec 5 2010Dec 8 2010

Publication series

NameProceedings - Winter Simulation Conference
ISSN (Print)0891-7736


Other2010 43rd Winter Simulation Conference, WSC'10
Country/TerritoryUnited States
CityBaltimore, MD

All Science Journal Classification (ASJC) codes

  • Software
  • Modeling and Simulation
  • Computer Science Applications


Dive into the research topics of 'Optimal learning of transition probabilities in the two-agent newsvendor problem'. Together they form a unique fingerprint.

Cite this