P -CURVING AS A SAFEGUARD AGAINST P -HACKING IN SLA RESEARCH A CASE STUDY

• Published in: Studies in second language acquisition Vol. 44; no. 4; pp. 1155 - 1180
• Main Author: Lindstromberg, Seth
• Format: Journal Article
• Language: English
• Published: New York Cambridge University Press 01.09.2022
• Subjects: Asymmetry; Bias; Data Analysis; Effect Size; Hypotheses; Language Proficiency; Meta Analysis; Methods; Native Language; Reading; Research Design; Research Methodology; Researchers; Resistance (Psychology); Second Language Learning; Second language vocabulary learning; Statistical Significance; Vocabulary Development
• ISSN: 0272-2631; 1470-1545
• DOI: 10.1017/S0272263121000516

It is important to be able to identify research results likely to have been arrived at by means of “ p -hacking,” a common term for research and reporting practices (such as the selective reporting of results) that are biased toward finding p < α. This paper discusses and demonstrates “ p -curving,” a means of checking a set of primary studies within a specific research stream for signs of p -hacking. A salient feature of p -curving is that it is based entirely on significant p -values. Because of the potential usefulness of p -curving and because it has been little used by SLA researchers, a case study illustrates the construction and analysis of a p -curve as a complement to meta-analysis. The focal p -curve in this study relates to published (quasi)experimental studies that addressed the research hypothesis that for low and middle proficiency learners L1 glosses facilitate vocabulary learning during reading better than L2 glosses do.