Optimal learning for stochastic optimization with nonlinear parametric belief models

Xinyu He, Warren Buckler Powell

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


We consider the problem of estimating the expected value of information (the knowledge gradient) for Bayesian learning problems where the belief model is nonlinear in the parameters.Our goal is to maximize an objective function represented by a nonlinear parametric belief model,while simultaneously learning the unknown parameters, by guiding a sequential experimentationprocess which is expensive. We overcome the problem of computing the expected value of an experiment, which is computationally intractable, by using a sampled approximation, which helps toguide experiments but does not provide an accurate estimate of the unknown parameters. We thenintroduce a resampling process which allows the sampled model to adapt to new information, exploiting past experiments. We show theoretically that the method generates sequences that convergeasymptotically to the true parameters, while simultaneously maximizing the objective function. Weshow empirically that the process exhibits rapid convergence, yielding good results with a very smallnumber of experiments.

Original languageEnglish (US)
Pages (from-to)2327-2359
Number of pages33
JournalSIAM Journal on Optimization
Issue number3
StatePublished - 2018

All Science Journal Classification (ASJC) codes

  • Software
  • Theoretical Computer Science
  • Applied Mathematics


  • Knowledge gradient
  • Nonlinear parametric model
  • Optimal learning


Dive into the research topics of 'Optimal learning for stochastic optimization with nonlinear parametric belief models'. Together they form a unique fingerprint.

Cite this