The randomized marker method for single-case randomization tests: Handling data missing at random and data missing not at random

• Published in: Behavior research methods Vol. 54; no. 6; pp. 2905 - 2938
• Main Authors: De, Tamal Kumar, Onghena, Patrick
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.12.2022; Springer Nature B.V
• Subjects: Behavioral Science and Psychology; Cognitive Psychology; Experiments; Missing data; Psychology; Statistical power; Statistics; Missing data; Randomization tests; Simulation; Single-case; Statistical power
• ISSN: 1554-3528; 1554-3528
• DOI: 10.3758/s13428-021-01781-5

Single-case experiments are frequently plagued by missing data problems. In a recent study, the randomized marker method was found to be valid and powerful for single-case randomization tests when the missing data were missing completely at random. However, in real-life experiments, it is difficult for researchers to ascertain the missing data mechanism. For analyzing such experiments, it is essential that the missing data handling method is valid and powerful for various missing data mechanisms. Hence, we examined the performance of the randomized marker method for data that are missing at random and data that are missing not at random. In addition, we compared the randomized marker method with multiple imputation, because the latter is often considered the gold standard among imputation techniques. To compare and evaluate these two methods under various simulation conditions, we calculated the type I error rate and statistical power in single-case randomization tests using these two methods of handling missing data and compared them to the type I error rate and statistical power using complete datasets. The results indicate that while multiple imputation presents an advantage in the presence of strongly correlated covariate data, the randomized marker method remains valid and results in sufficient statistical power for most of the missing data conditions simulated in this study.