TY - JOUR
T1 - Parallelograms revisited
T2 - Exploring the limitations of vector space models for simple analogies
AU - Peterson, Joshua C.
AU - Chen, Dawn
AU - Griffiths, Thomas L.
N1 - Funding Information:
This work was supported by grant number FA9550-13-1-0170 from the Air Force Office of Scientific Research and grant number 1718550 from the National Science Foundation . Preliminary results for Experiments 2-4 were presented at the Annual Conference of the Cognitive Science Society ( Chen et al., 2017 ).
Publisher Copyright:
© 2020 Elsevier B.V.
PY - 2020/12
Y1 - 2020/12
N2 - Classic psychological theories have demonstrated the power and limitations of spatial representations, providing geometric tools for reasoning about the similarity of objects and showing that human intuitions sometimes violate the constraints of geometric spaces. Recent machine learning methods for deriving vector-space embeddings of words have begun to garner attention for their surprising capacity to capture simple analogies consistently across large corpora, giving new life to a classic model of analogies as parallelograms that was first proposed and briefly explored by psychologists. We evaluate the parallelogram model of analogy as applied to modern data-driven word embeddings, providing a detailed analysis of the extent to which this approach captures human behavior in the domain of word pairs. Using a large novel benchmark dataset of human analogy completions, we show that word similarity alone surprisingly captures some aspects of human responses better than the parallelogram model. To gain a fine-grained picture of how well these models predict relational similarity, we also collect a large dataset of human relational similarity judgments and find that the parallelogram model captures some semantic relationships better than others. Finally, we provide evidence for deeper limitations of the parallelogram model of analogy based on the intrinsic geometric constraints of vector spaces, paralleling classic results for item similarity. Taken together, these results show that while modern word embeddings do an impressive job of capturing semantic similarity at scale, the parallelogram model alone is insufficient to account for how people form even the simplest analogies.
AB - Classic psychological theories have demonstrated the power and limitations of spatial representations, providing geometric tools for reasoning about the similarity of objects and showing that human intuitions sometimes violate the constraints of geometric spaces. Recent machine learning methods for deriving vector-space embeddings of words have begun to garner attention for their surprising capacity to capture simple analogies consistently across large corpora, giving new life to a classic model of analogies as parallelograms that was first proposed and briefly explored by psychologists. We evaluate the parallelogram model of analogy as applied to modern data-driven word embeddings, providing a detailed analysis of the extent to which this approach captures human behavior in the domain of word pairs. Using a large novel benchmark dataset of human analogy completions, we show that word similarity alone surprisingly captures some aspects of human responses better than the parallelogram model. To gain a fine-grained picture of how well these models predict relational similarity, we also collect a large dataset of human relational similarity judgments and find that the parallelogram model captures some semantic relationships better than others. Finally, we provide evidence for deeper limitations of the parallelogram model of analogy based on the intrinsic geometric constraints of vector spaces, paralleling classic results for item similarity. Taken together, these results show that while modern word embeddings do an impressive job of capturing semantic similarity at scale, the parallelogram model alone is insufficient to account for how people form even the simplest analogies.
KW - Analogy
KW - Relational similarity
KW - Vector space models
UR - http://www.scopus.com/inward/record.url?scp=85089956183&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85089956183&partnerID=8YFLogxK
U2 - 10.1016/j.cognition.2020.104440
DO - 10.1016/j.cognition.2020.104440
M3 - Article
C2 - 32882470
AN - SCOPUS:85089956183
SN - 0010-0277
VL - 205
JO - Cognition
JF - Cognition
M1 - 104440
ER -