The origins of unpredictability in life outcome prediction tasks

Ian Lundberg, Rachel Brown-Weinstock, Susan Clampet-Lundquist, Sarah Pachman, Timothy J. Nelson, Vicki Yang, Kathryn Edin, Matthew J. Salganik

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Why are some life outcomes difficult to predict? We investigated this question through in-depth qualitative interviews with 40 families sampled from a multidecade longitudinal study. Our sampling and interviewing process was informed by the earlier efforts of hundreds of researchers to predict life outcomes for participants in this study. The qualitative evidence we uncovered in these interviews combined with a mathematical decomposition of prediction error led us to create a conceptual framework. Our specific evidence and our more general framework suggest that unpredictability should be expected in many life outcome prediction tasks, even in the presence of complex algorithms and large datasets. Our work provides a foundation for future empirical and theoretical work on unpredictability in human lives.

Original languageEnglish (US)
Article numbere2322973121
JournalProceedings of the National Academy of Sciences of the United States of America
Volume121
Issue number24
DOIs
StatePublished - Jun 11 2024

All Science Journal Classification (ASJC) codes

  • General

Keywords

  • life course
  • limits to prediction
  • machine learning
  • mixed methods
  • prediction

Fingerprint

Dive into the research topics of 'The origins of unpredictability in life outcome prediction tasks'. Together they form a unique fingerprint.

Cite this