Sample Size Calculation and Optimal Design for Multivariate Regression-Based Norming

• Published in: Journal of educational and behavioral statistics Vol. 49; no. 5; pp. 817 - 847
• Main Authors: Innocenti, Francesco, Candel, Math J. J. M., Tan, Frans E. S., van Breukelen, Gerard J. P.
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.10.2024; American Educational Research Association
• Subjects: Computation; Educational tests & measurements; Sample Size; optimal design; Z-score; sample size calculation; normative data; Mahalanobis distance
• DOI: 10.3102/10769986231210807
• ISSN: 1076-9986

Normative studies are needed to obtain norms for comparing individuals with the reference population on relevant clinical or educational measures. Norms can be obtained in an efficient way by regressing the test score on relevant predictors, such as age and sex. When several measures are normed with the same sample, a multivariate regression-based approach must be adopted for at least two reasons: (1) to take into account the correlations between the measures of the same subject, in order to test certain scientific hypotheses and to reduce misclassification of subjects in clinical practice, and (2) to reduce the number of significance tests involved in selecting predictors for the purpose of norming, thus preventing the inflation of the type I error rate. A new multivariate regression-based approach is proposed that combines all measures for an individual through the Mahalanobis distance, thus providing an indicator of the individual’s overall performance. Furthermore, optimal designs for the normative study are derived under five multivariate polynomial regression models, assuming multivariate normality and homoscedasticity of the residuals, and efficient robust designs are presented in case of uncertainty about the correct model for the analysis of the normative sample. Sample size calculation formulas are provided for the new Mahalanobis distance-based approach. The results are illustrated with data from the Maastricht Aging Study (MAAS).