Hierarchical knowledge gradient for sequential sampling

Martijn R.K. Mes, Warren Buckler Powell, Peter I. Frazier

Research output: Contribution to journalArticlepeer-review

19 Scopus citations


We propose a sequential sampling policy for noisy discrete global optimization and ranking and selection, in which we aim to efficiently explore a finite set of alternatives before selecting an alternative as best when exploration stops. Each alternative may be characterized by a multidimensional vector of categorical and numerical attributes and has independent normal rewards. We use a Bayesian probability model for the unknown reward of each alternative and follow a fully sequential sampling policy called the knowledge-gradient policy. This policy myopically optimizes the expected increment in the value of sampling information in each time period. We propose a hierarchical aggregation technique that uses the common features shared by alternatives to learn about many alternatives from even a single measurement. This approach greatly reduces the measurement effort required, but it requires some prior knowledge on the smoothness of the function in the form of an aggregation function and computational issues limit the number of alternatives that can be easily considered to the thousands. We prove that our policy is consistent, finding a globally optimal alternative when given enough measurements, and show through simulations that it performs competitively with or significantly better than other policies.

Original languageEnglish (US)
Pages (from-to)2931-2974
Number of pages44
JournalJournal of Machine Learning Research
StatePublished - Oct 2011

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence
  • Control and Systems Engineering
  • Statistics and Probability


  • Adaptive learning
  • Bayesian statistics
  • Hierarchical statistics
  • Ranking and selection
  • Sequential experimental design


Dive into the research topics of 'Hierarchical knowledge gradient for sequential sampling'. Together they form a unique fingerprint.

Cite this