Value function approximation using multiple aggregation for multiattribute resource management

Abraham George, Warren B. Powell, Sanjeev R. Kulkarni

Research output: Contribution to journalArticlepeer-review

41 Scopus citations

Abstract

We consider the problem of estimating the value of a multiattribute resource, where the attributes are categorical or discrete in nature and the number of potential attribute vectors is very large. The problem arises in approximate dynamic programming when we need to estimate the value of a multiattribute resource from estimates based on Monte-Carlo simulation. These problems have been traditionally solved using aggregation, but choosing the right level of aggregation requires resolving the classic tradeoff between aggregation error and sampling error. We propose a method that estimates the value of a resource at different levels of aggregation simultaneously, and then uses a weighted combination of the estimates. Using the optimal weights, which minimizes the variance of the estimate while accounting for correlations between the estimates, is computationally too expensive for practical applications. We have found that a simple inverse variance formula (adjusted for bias), which effectively assumes the estimates are independent, produces near-optimal estimates. We use the setting of two levels of aggregation to explain why this approximation works so well.

Original languageEnglish (US)
Pages (from-to)2079-2111
Number of pages33
JournalJournal of Machine Learning Research
Volume9
StatePublished - Oct 2008

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence
  • Control and Systems Engineering
  • Statistics and Probability

Keywords

  • Adaptive learning
  • Approximate dynamic programming
  • Hierarchical statistics
  • Mixture models
  • Multiattribute resources

Fingerprint

Dive into the research topics of 'Value function approximation using multiple aggregation for multiattribute resource management'. Together they form a unique fingerprint.

Cite this