The viability of crowdsourcing for survey research

• Published in: Behavior research methods Vol. 43; no. 3; pp. 800 - 813
• Main Authors: Behrend, Tara S., Sharek, David J., Meade, Adam W., Wiebe, Eric N.
• Format: Journal Article
• Language: English
• Published: New York Springer-Verlag 01.09.2011; Springer Nature B.V
• Subjects: Behavioral Research; Behavioral Science and Psychology; Cognitive Psychology; Compensation; Crowdsourcing; Data collection; Data Collection - methods; Humans; Internet; Patient Selection; Personality; Psychology; Research Design; Research Subjects; Sampling techniques; Studies; Wages & salaries; Workforce; Crowdsourcing; Survey research; Industrial/organizational psychology; Personality; Sampling; Mechanical Turk
• ISSN: 1554-3528; 1554-351X; 1554-3528
• DOI: 10.3758/s13428-011-0081-0

Online contract labor portals (i.e., crowdsourcing) have recently emerged as attractive alternatives to university participant pools for the purposes of collecting survey data for behavioral research. However, prior research has not provided a thorough examination of crowdsourced data for organizational psychology research. We found that, as compared with a traditional university participant pool, crowdsourcing respondents were older, were more ethnically diverse, and had more work experience. Additionally, the reliability of the data from the crowdsourcing sample was as good as or better than the corresponding university sample. Moreover, measurement invariance generally held across these groups. We conclude that the use of these labor portals is an efficient and appropriate alternative to a university participant pool, despite small differences in personality and socially desirable responding across the samples. The risks and advantages of crowdsourcing are outlined, and an overview of practical and ethical guidelines is provided.