Effect of variance ratio on ANOVA robustness: Might 1.5 be the limit?

• Published in: Behavior research methods Vol. 50; no. 3; pp. 937 - 962
• Main Authors: Blanca, María J., Alarcón, Rafael, Arnau, Jaume, Bono, Roser, Bendayan, Rebecca
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.06.2018
• Subjects: Behavioral Science and Psychology; Cognitive Psychology; Psychology; Variance ratio; Heterogeneity; F-test; Robustness; ANOVA
• ISSN: 1554-3528; 1554-3528
• DOI: 10.3758/s13428-017-0918-2

Inconsistencies in the research findings on F -test robustness to variance heterogeneity could be related to the lack of a standard criterion to assess robustness or to the different measures used to quantify heterogeneity. In the present paper we use Monte Carlo simulation to systematically examine the Type I error rate of F -test under heterogeneity. One-way, balanced, and unbalanced designs with monotonic patterns of variance were considered. Variance ratio (VR) was used as a measure of heterogeneity (1.5, 1.6, 1.7, 1.8, 2, 3, 5, and 9), the coefficient of sample size variation as a measure of inequality between group sizes (0.16, 0.33, and 0.50), and the correlation between variance and group size as an indicator of the pairing between them (1, .50, 0, −.50, and −1). Overall, the results suggest that in terms of Type I error a VR above 1.5 may be established as a rule of thumb for considering a potential threat to F -test robustness under heterogeneity with unequal sample sizes.