Paying Attention to Inattentive Survey Respondents

• Published in: Political analysis Vol. 27; no. 2; pp. 145 - 162
• Main Authors: Alvarez, R. Michael, Atkeson, Lonna Rae, Levin, Ines, Li, Yimeng
• Format: Journal Article
• Language: English
• Published: New York, USA Cambridge University Press 01.04.2019
• Subjects: Bias; Data quality; Deflation; Experiments; Internet; Political attitudes; Political behavior; Political science; Polls & surveys; Researchers; Respondents; Responses; instructed-response items; trap questions; sensitive questions; survey design; survey response; list experiments; satisficing
• ISSN: 1047-1987; 1476-4989
• DOI: 10.1017/pan.2018.57

Does attentiveness matter in survey responses? Do more attentive survey participants give higher quality responses? Using data from a recent online survey that identified inattentive respondents using instructed-response items, we demonstrate that ignoring attentiveness provides a biased portrait of the distribution of critical political attitudes and behavior. We show that this bias occurs in the context of both typical closed-ended questions and in list experiments. Inattentive respondents are common and are more prevalent among the young and less educated. Those who do not pass the trap questions interact with the survey instrument in distinctive ways: they take less time to respond; are more likely to report nonattitudes; and display lower consistency in their reported choices. Inattentiveness does not occur completely at random and failing to properly account for it may lead to inaccurate estimates of the prevalence of key political attitudes and behaviors, of both sensitive and more prosaic nature.