An item response theory analysis of the matrix reasoning item bank (MaRs-IB)

Samuel Zorowitz, Gabriele Chierchia, Sarah Jayne Blakemore, Nathaniel D. Daw

Research output: Contribution to journalArticlepeer-review

Abstract

Matrix reasoning tasks are among the most widely used measures of cognitive ability in the behavioral sciences, but the lack of matrix reasoning tests in the public domain complicates their use. Here, we present an extensive investigation and psychometric validation of the matrix reasoning item bank (MaRs-IB), an open-access set of matrix reasoning items. In a first study, we calibrate the psychometric functioning of the items in the MaRs-IB in a large sample of adult participants (N = 1501). Using additive multilevel item structure models, we establish that the MaRs-IB has many desirable psychometric properties: its items span a wide range of difficulty, possess medium-to-large levels of discrimination, and exhibit robust associations between item complexity and difficulty. However, we also find that item clones are not always psychometrically equivalent and cannot be assumed to be exchangeable. In a second study, we demonstrate how experimenters can use the estimated item parameters to design new matrix reasoning tests using optimal item assembly. Specifically, we design and validate two new sets of test forms in an independent sample of adults (N = 600). We find these new tests possess good reliability and convergent validity with an established measure of matrix reasoning. We hope that the materials and results made available here will encourage experimenters to use the MaRs-IB in their research.

Original languageEnglish (US)
Pages (from-to)1104-1122
Number of pages19
JournalBehavior Research Methods
Volume56
Issue number3
DOIs
StatePublished - Mar 2024

All Science Journal Classification (ASJC) codes

  • Experimental and Cognitive Psychology
  • Developmental and Educational Psychology
  • Arts and Humanities (miscellaneous)
  • Psychology (miscellaneous)
  • General Psychology

Keywords

  • Item response theory
  • Matrix reasoning
  • Progressive matrices
  • Speed-accuracy trade-off

Fingerprint

Dive into the research topics of 'An item response theory analysis of the matrix reasoning item bank (MaRs-IB)'. Together they form a unique fingerprint.

Cite this