What to simulate? Inferring the right direction for mental rotation

When people use mental imagery, how do they decide which images to generate? To answer this question, we explored how mental simulation should be used in the classic psychological task of determining if two images depict the same object in different orientations (Shepard & Metzler, 1971). Through a rational analysis of mental rotation, we formalized four models and compared them to human performance. We found that three models based on previous hypotheses in the literature were unable to account for several aspects of human behavior. The fourth is based on the idea active sampling (e.g., Gureckis & Markant, 2012), which is a strategy of choosing actions that will provide the most information. This last model provides a plausible account of how people use mental rotation, where the other models do not. Based on these results, we suggest that the question of “what to simulate?” is more difficult than has previously been assumed, and that an active learning approach holds promise for uncovering the answer.