Selecting a Within- or Between-Subject Design for Mediation: Validity, Causality, and Statistical Power

• Published in: Multivariate behavioral research Vol. 58; no. 3; pp. 616 - 636
• Main Author: Montoya, Amanda K.
• Format: Journal Article
• Language: English
• Published: United States Routledge 04.05.2023; Taylor & Francis Ltd
• Subjects: causal inference; Causality; Design factors; Empirical analysis; indirect effect; Mediation analysis; Monte Carlo simulation; power; power analysis; Statistical power; type I error; Validity; within-subject design; indirect effect; power; validity; within-subject design; causal inference; Mediation analysis; Monte Carlo simulation; type I error; power analysis
• ISSN: 0027-3171; 1532-7906; 1532-7906
• DOI: 10.1080/00273171.2022.2077287

Researchers with mediation hypotheses must consider which design to use: within-subject or between-subject? In this paper, I argue that three factors should influence design choice: validity, causality, and statistical power. Threats to validity include carry-over effects, participant awareness, measurement, and more. Causality is a core element of mediation, and the assumptions required for causal inference differ between the two designs. Between-subject designs require more restrictive no-confounder assumptions, but within-subject designs require the assumption of no carry-over effects. Statistical power should be higher in within-subject designs, but the degree and conditions of this advantage are unknown for mediation analysis. A Monte Carlo simulation compares designs under a broad range of sample sizes, effect sizes, and correlations among repeated measurements. The results show within-subject designs require about half the sample size of between-subject designs to detect indirect effects of the same size, but this difference can vary with population parameters. I provide an empirical example and R script for conducting power analysis for within-subject mediation analysis. Researchers interested in conducting mediation analysis should not select within-subject designs merely because of higher power, but they should also consider validity and causality in their decision, both of which can favor between-subject designs.