Variable criteria sequential stopping rule: Validity and power with repeated measures ANOVA, multiple correlation, MANOVA and relation to Chi-square distribution

• Published in: Behavior research methods Vol. 50; no. 5; pp. 1988 - 2003
• Main Author: Fitts, Douglas A.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.10.2018
• Subjects: Analysis of Variance; Behavioral Science and Psychology; Chi-Square Distribution; Cognitive Psychology; Humans; Psychology; Reproducibility of Results; Research Design; Sample Size; Multiple correlation; Reduction of subjects; Group sequential stopping rules; Repeated measures; Sample size planning; Effect size; Friedman test; Power analysis; Chi square; MANOVA; Analysis of variance; Null hypothesis significance test
• ISSN: 1554-3528; 1554-3528
• DOI: 10.3758/s13428-017-0968-5

The variable criteria sequential stopping rule (vcSSR) is an efficient way to add sample size to planned ANOVA tests while holding the observed rate of Type I errors, α o , constant. The only difference from regular null hypothesis testing is that criteria for stopping the experiment are obtained from a table based on the desired power, rate of Type I errors, and beginning sample size. The vcSSR was developed using between-subjects ANOVAs, but it should work with p values from any type of F test. In the present study, the α o remained constant at the nominal level when using the previously published table of criteria with repeated measures designs with various numbers of treatments per subject, Type I error rates, values of ρ, and four different sample size models. New power curves allow researchers to select the optimal sample size model for a repeated measures experiment. The criteria held α o constant either when used with a multiple correlation that varied the sample size model and the number of predictor variables, or when used with MANOVA with multiple groups and two levels of a within-subject variable at various levels of ρ. Although not recommended for use with χ 2 tests such as the Friedman rank ANOVA test, the vcSSR produces predictable results based on the relation between F and χ 2 . Together, the data confirm the view that the vcSSR can be used to control Type I errors during sequential sampling with any t - or F -statistic rather than being restricted to certain ANOVA designs.