Predicting Human Similarity Judgments Using Large Language Models

Raja Marjieh, Ilia Sucholutsky, Theodore R. Sumers, Nori Jacoby, Thomas L. Griffiths

Research output: Contribution to conferencePaperpeer-review

Abstract

Similarity judgments provide a well-established method for accessing mental representations, with applications in psychology, neuroscience and machine learning. However, collecting similarity judgments can be prohibitively expensive for naturalistic datasets as the number of comparisons grows quadratically in the number of stimuli. One way to tackle this problem is to construct approximation procedures that rely on more accessible proxies for predicting similarity. Here we leverage recent advances in language models and online recruitment, proposing an efficient domain-general procedure for predicting human similarity judgments based on text descriptions. Intuitively, similar stimuli are likely to evoke similar descriptions, allowing us to use description similarity to predict pairwise similarity judgments. Crucially, the number of descriptions required grows only linearly with the number of stimuli, drastically reducing the amount of data required. We test this procedure on six datasets of naturalistic images and show that our models outperform previous approaches based on visual information.

Original languageEnglish (US)
Pages2598-2604
Number of pages7
StatePublished - 2022
Event44th Annual Meeting of the Cognitive Science Society: Cognitive Diversity, CogSci 2022 - Toronto, Canada
Duration: Jul 27 2022Jul 30 2022

Conference

Conference44th Annual Meeting of the Cognitive Science Society: Cognitive Diversity, CogSci 2022
Country/TerritoryCanada
CityToronto
Period7/27/227/30/22

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Science Applications
  • Human-Computer Interaction
  • Cognitive Neuroscience

Keywords

  • language models
  • perception
  • representations
  • similarity

Fingerprint

Dive into the research topics of 'Predicting Human Similarity Judgments Using Large Language Models'. Together they form a unique fingerprint.

Cite this