Comparing online and lab methods in a problem-solving experiment

• Published in: Behavior research methods Vol. 40; no. 2; pp. 428 - 434
• Main Authors: Dandurand, Frédéric, Shultz, Thomas R., Onishi, Kristine H.
• Format: Journal Article
• Language: English
• Published: New York Springer-Verlag 01.05.2008; Psychonomic Society, Inc
• Subjects: Adult; Behavioral Science and Psychology; Clinical psychology; Cognitive Psychology; Data Collection - methods; Environment; Experiments; Female; Humans; Internet; Learning; Mailing lists; Male; Monetary incentives; Personal information; Problem Solving; Psychology; Psychology, Experimental - methods; Psychometrics - methods; Reference Values; Software; Studies; Task Performance and Analysis; Explicit Learning; Online Participant; Subject Pool; Online Experiment; Behavior Research Method
• ISSN: 1554-351X; 1554-3528
• DOI: 10.3758/BRM.40.2.428

Online experiments have recently become very popular, and—in comparison with traditional lab experiments— they may have several advantages, such as reduced demand characteristics, automation, and generalizability of results to wider populations (Birnbaum, 2004; Reips, 2000, 2002a, 2002b). We replicated Dandurand, Bowen, and Shultz’s (2004) lab-based problem-solving experiment as an Internet experiment. Consistent with previous results, we found that participants who watched demonstrations of successful problem-solving sessions or who read instructions outperformed those who were told only that they solved problems correctly or not. Online participants were less accurate than lab participants, but there was no interaction with learning condition. Thus, we conclude that online and Internet results are consistent. Disadvantages included high dropout rate for online participants; however, combining the online experiment with the department subject pool worked well.