Embedding Regression: Models for Context-Specific Description and Inference

Pedro L. Rodriguez, Arthur Spirling, Brandon M. Stewart

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

Social scientists commonly seek to make statements about how word use varies over circumstances-including time, partisan identity, or some other document-level covariate. For example, researchers might wish to know how Republicans and Democrats diverge in their understanding of the term immigration. Building on the success of pretrained language models, we introduce the à la carte on text (conText) embedding regression model for this purpose. This fast and simple method produces valid vector representations of how words are used-and thus what words mean-in different contexts. We show that it outperforms slower, more complicated alternatives and works well even with very few documents. The model also allows for hypothesis testing and statements about statistical significance. We demonstrate that it can be used for a broad range of important tasks, including understanding US polarization, historical legislative development, and sentiment detection. We provide open-source software for fitting the model.

Original languageEnglish (US)
Pages (from-to)1255-1274
Number of pages20
JournalAmerican Political Science Review
Volume117
Issue number4
DOIs
StatePublished - Nov 19 2023

All Science Journal Classification (ASJC) codes

  • Sociology and Political Science
  • Political Science and International Relations

Fingerprint

Dive into the research topics of 'Embedding Regression: Models for Context-Specific Description and Inference'. Together they form a unique fingerprint.

Cite this