From convolutional neural networks to models of higher-level cognition (and back again)

Ruairidh M. Battleday, Joshua C. Peterson, Thomas L. Griffiths

Research output: Contribution to journalReview articlepeer-review

16 Scopus citations


The remarkable successes of convolutional neural networks (CNNs) in modern computer vision are by now well known, and they are increasingly being explored as computational models of the human visual system. In this paper, we ask whether CNNs might also provide a basis for modeling higher-level cognition, focusing on the core phenomena of similarity and categorization. The most important advance comes from the ability of CNNs to learn high-dimensional representations of complex naturalistic images, substantially extending the scope of traditional cognitive models that were previously only evaluated with simple artificial stimuli. In all cases, the most successful combinations arise when CNN representations are used with cognitive models that have the capacity to transform them to better fit human behavior. One consequence of these insights is a toolkit for the integration of cognitively motivated constraints back into CNN training paradigms in computer vision and machine learning, and we review cases where this leads to improved performance. A second consequence is a roadmap for how CNNs and cognitive models can be more fully integrated in the future, allowing for flexible end-to-end algorithms that can learn representations from data while still retaining the structured behavior characteristic of human cognition.

Original languageEnglish (US)
Pages (from-to)55-78
Number of pages24
JournalAnnals of the New York Academy of Sciences
Issue number1
StatePublished - Dec 2021

All Science Journal Classification (ASJC) codes

  • General Biochemistry, Genetics and Molecular Biology
  • General Neuroscience
  • History and Philosophy of Science


Dive into the research topics of 'From convolutional neural networks to models of higher-level cognition (and back again)'. Together they form a unique fingerprint.

Cite this