Who watches presidential debates? Measurement problems in campaign effects research

To examine whether a campaign event affected candidate preferences or candidate knowledge, survey researchers need to know who was exposed to the event. Using the example of presidential debates, this study shows that survey respondents do not accurately report their exposure to even the most salient campaign events. Two independent methods are used to assess the validity of self-reported debate exposure. First, survey estimates are compared to Nielsen estimates, which track exposure automatically. Second, the temporal stability of self-reports across independent daily estimates in the National Annenberg Election Survey is analyzed. Both approaches indicate low validity of self-reports. Self-reported debate audiences are approximately twice as big as comparable Nielsen estimates. Independent random samples generate widely divergent audience estimates for the same debate depending on when the survey was conducted. The second finding demonstrates low validity of self-reports without assuming validity of Nielsen estimates (or any other benchmark). The low validity of self-reported debate exposure poses a major obstacle for campaign effects research. Without valid measures of who was exposed to a campaign event, research cannot establish the causal impact of the event.