Insufficient effort responding: examining an insidious confound in survey data

• Published in: Journal of applied psychology Vol. 100; no. 3; p. 828
• Main Authors: Huang, Jason L, Liu, Mengqiao, Bowling, Nathan A
• Format: Journal Article
• Language: English
• Published: United States 01.05.2015
• Subjects: Adult; Employment - psychology; Female; Humans; Male; Middle Aged; Personality Assessment - standards; Psychometrics - standards; Self Report - standards; Surveys and Questionnaires - standards; Young Adult
• ISSN: 1939-1854
• DOI: 10.1037/a0038510

Insufficient effort responding (IER; Huang, Curran, Keeney, Poposki, & DeShon, 2012) to surveys has largely been assumed to be a source of random measurement error that attenuates associations between substantive measures. The current article, however, illustrates how and when the presence of IER can produce a systematic bias that inflates observed correlations between substantive measures. Noting that inattentive responses as a whole generally congregate around the midpoint of a Likert scale, we propose that Mattentive, defined as the mean score of attentive respondents on a substantive measure, will be negatively related to IER's confounding effect on substantive measures (i.e., correlations between IER and a given substantive measure will become less positive [or more negative] as Mattentive increases). Results from a personality questionnaire (Study 1) and a simulation (Study 2) consistently support the hypothesized confounding influence of IER. Using an employee sample (Study 3), we demonstrated how IER can confound bivariate relationships between substantive measures. Together, these studies indicate that IER can inflate the strength of observed relationships when scale means depart from the scale midpoints, resulting in an inflated Type I error rate. This challenges the traditional view that IER attenuates observed bivariate correlations. These findings highlight situations where IER may be a methodological nuisance, while underscoring the need for survey administrators and researchers to deter and detect IER in surveys. The current article serves as a wake-up call for researchers and practitioners to more closely examine IER in their data.