Model-based adaptive critic designs

Silvia Ferrari, Robert F. Stengel

Research output: Chapter in Book/Report/Conference proceedingChapter

45 Scopus citations


This chapter provides an overview of model-based adaptive critic designs including background, general algorithms, implementations, and comparisons. The authors begin by introducing the mathematical background of model-reference adaptive critic designs. Various ADP designs such as Heuristic Dynamic Programming (HDP), Dual HDP (DHP), Globalized DHP (GDHP), and Action-Dependent (AD) designs are examined from both a mathematical and implementation standpoint and put into perspective. Pseudocode is provided for many aspects of the algorithms. The chapter concludes with applications and examples. For another overview perspective that focuses more on implementation issues read Chapter 4: Guidance in the Use of Adaptive Critics for Control. Chapter 15 contains a comparison of DHP with back-propagation through time, building a common framework for comparing these methods.

Original languageEnglish (US)
Title of host publicationHandbook of Learning and Approximate Dynamic Programming
PublisherWiley-IEEE Press
Number of pages31
ISBN (Electronic)9780470544785
ISBN (Print)047166054X, 9780471660545
StatePublished - Jan 1 2004

All Science Journal Classification (ASJC) codes

  • General Computer Science


  • Adaptation model
  • Dynamic programming
  • Equations
  • Heuristic algorithms
  • Mathematical model
  • Optimal control
  • Trajectory


Dive into the research topics of 'Model-based adaptive critic designs'. Together they form a unique fingerprint.

Cite this