Inaccurate forecasting of a randomized controlled trial

Mats Ahrenshop, Miriam Golden, Saad Gulzar, Luke Sonnet

Research output: Contribution to journalArticlepeer-review

Abstract

We report the results of a forecasting experiment about a randomized controlled trial that was conducted in the field. The experiment asks Ph.D. students, faculty, and policy practitioners to forecast (1) compliance rates for the RCT and (2) treatment effects of the intervention. The forecasting experiment randomizes the order of questions about compliance and treatment effects and the provision of information that a pilot experiment had been conducted which produced null results. Forecasters were excessively optimistic about treatment effects and unresponsive to item order as well as to information about a pilot. Those who declare themselves expert in the area relevant to the intervention are particularly resistant to new information that the treatment is ineffective. We interpret our results as suggesting that we should exercise caution when undertaking expert forecasting, since experts may have unrealistic expectations and may be inflexible in altering these even when provided new information.

Original languageEnglish (US)
JournalJournal of Experimental Political Science
DOIs
StateAccepted/In press - 2023
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Sociology and Political Science

Keywords

  • Experts
  • Forecasting
  • ICT
  • Information
  • RCTs

Fingerprint

Dive into the research topics of 'Inaccurate forecasting of a randomized controlled trial'. Together they form a unique fingerprint.

Cite this