A primer on the validity typology and threats to validity in education research

• Published in: Asia Pacific education review Vol. 25; no. 3; pp. 557 - 574
• Main Authors: Anglin, Kylie, Liu, Qing, Wong, Vivian C.
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01.09.2024; Springer Nature B.V
• Subjects: Causality; Construct Validity; Education; Educational Practices; Educational Research; Inferences; Validity; Causal inference; Validity; Program evaluation; Impact evaluation
• ISSN: 1598-1037; 1876-407X
• DOI: 10.1007/s12564-024-09955-4

Given decision-makers often prioritize causal research that identifies the impact of treatments on the people they serve, a key question in education research is, “Does it work?”. Today, however, researchers are paying increasing attention to successive questions that are equally important from a practical standpoint—not only does it work, but for whom and under what circumstances? Invalid conclusions to any of these questions can result in the adoption of ineffective educational practices. This article discusses the enduring legacy of Shadish, Cook, and Campbell’s validity typology, and its associated threats to validity, for improving the validity of inferences in education research. The validity typology provides a system for classifying and improving inferences related to four validity types, including ensuring a causal relationship between a treatment and outcome (internal validity) that is precisely estimated (statistical validity), well understood (construct validity), and generalizes to the necessary circumstances (external validity). Here, we provide an overview of these four validity types and discuss proactive approaches to addressing them. We conclude by discussing how the validity typology framework may help researchers understand and address contemporary critiques of quantitative causal research.