A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples

• Published in: Journal of business research Vol. 69; no. 8; pp. 3139 - 3148
• Main Authors: Smith, Scott M., Roster, Catherine A., Golden, Linda L., Albaum, Gerald S.
• Format: Journal Article
• Language: English
• Published: New York Elsevier Inc 01.08.2016; Elsevier Sequoia S.A
• Subjects: Consumer panels; Data collection; Data quality; Internet; Internet surveys; Mechanical Turk; Polls & surveys; Quality; Respondents; Responses; Studies; Survey research; United States > US; Respondents; Survey research; Data quality; Consumer panels; Mechanical Turk; Internet surveys
• ISSN: 0148-2963; 1873-7978
• DOI: 10.1016/j.jbusres.2015.12.002

With the exploding use of Internet surveys, research efforts and data quality are increasingly subject to the effects of respondents who do not give the required attention to survey questions and who speed through the survey, or who intentionally cheat with their answers. We investigate respondent integrity and data quality for samples drawn from a “Regular” online panel and from Amazon's MTurk. New metrics for assessing sample integrity and online data quality are introduced. Overall, MTurk respondents in both respondent groups took less time to answer questions. The non-USA MTurk group deviated most from correct answers in attention filter questions and had more duplicate IP addresses. In addition, the results from the three Internet sample sources are substantively different. The choice of an Internet survey sample vendor is critical, as it can impact sample composition, respondent integrity, data quality, data structure and substantive results.