Strengthening the Experimenter's Toolbox: Statistical Estimation of Internal Validity

Luke Keele, Corrine McConnaughy, Ismail White

Research output: Contribution to journalArticlepeer-review

35 Scopus citations


Experiments have become an increasingly common tool for political science researchers over the last decade, particularly laboratory experiments performed on small convenience samples. We argue that the standard normal theory statistical paradigm used in political science fails to meet the needs of these experimenters and outline an alternative approach to statistical inference based on randomization of the treatment. The randomization inference approach not only provides direct estimation of the experimenter's quantity of interest-the certainty of the causal inference about the observed units-but also helps to deal with other challenges of small samples. We offer an introduction to the logic of randomization inference, a brief overview of its technical details, and guidance for political science experimenters about making analytic choices within the randomization inference framework. Finally, we reanalyze data from two political science experiments using randomization tests to illustrate the inferential differences that choosing a randomization inference approach can make.

Original languageEnglish (US)
Pages (from-to)484-499
Number of pages16
JournalAmerican Journal of Political Science
Issue number2
StatePublished - Apr 2012
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Sociology and Political Science
  • Political Science and International Relations


Dive into the research topics of 'Strengthening the Experimenter's Toolbox: Statistical Estimation of Internal Validity'. Together they form a unique fingerprint.

Cite this