How to show that a cruel prank is worse than a war crime: Shifting scales and missing benchmarks in the study of moral judgment

Moral judgment is central to both everyday life and cognitive science, but how can it be studied with quantitative precision? By far the most direct and ubiquitous method is to simply ask people for their judgments, in the form of ratings on a labeled scale (e.g. Likert or Visual Analog Scales). As has long been recognized in sensory psychophysics, however, such responses are meaningful only in a relative sense. (Is your dog “big”? Perhaps yes in the context of house pets, but not in the context of all mammals?) Here we illustrate the nature and extremity of this problem using two case studies. First, to explore this theme in principle, we show in a series of nine experiments that this problem can readily lead subjects to (seemingly) judge a cruel prank (involving humiliation) to be just as immoral as (or even worse than) an internationally recognized war crime (involving murder). In contrast, such nonsensical results disappear when using magnitude estimation — a psychophysical method employing an explicit moral benchmark. Second, to demonstrate the importance of this theme in practice, we show that the use of magnitude estimation (vs. Likert scales) radically changes the proper interpretation of a recent study of ‘moral luck’, fueling essentially the opposite conclusion. Taken together, this work illustrates how insights from psychophysics can help improve measurement in contemporary moral psychology.