Comparison of Multiple-Indicators, Multiple-Causes– and Item Response Theory–Based Analyses of Subgroup Differences

• Published in: Educational and psychological measurement Vol. 68; no. 4; pp. 587 - 602
• Main Authors: Willse, John T., Goodman, Joshua T.
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.08.2008; Sage; SAGE PUBLICATIONS, INC
• Subjects: Ability tests; Biological and medical sciences; Comparative Analysis; Differences; Educational evaluation; Effect Size; Fundamental and applied biological sciences. Psychology; Item Response Theory; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Psychometrics; Psychometrics. Statistics. Methodology; Raw Scores; Standardized tests; Statistics. Mathematics; Structural Equation Models; Test Length; Theory; effect size; structural equation modeling; item response theory; Statistical method; Structural equation modeling; Effect size; Item response theory; Psychometrics; Subgroup; Indicator
• ISSN: 0013-1644; 1552-3888
• DOI: 10.1177/0013164407312601

This research provides a direct comparison of effect size estimates based on structural equation modeling (SEM), item response theory (IRT), and raw scores. Differences between the SEM, IRT, and raw score approaches are examined under a variety of data conditions (IRT models underlying the data, test lengths, magnitude of group differences, and relative size of reference and focal groups). Results show that all studied methods perform similarly. All methods tend to underestimate effects as effect sizes become larger. SEM-based approaches to effect size estimation perform somewhat better at shorter test lengths, whereas approaches based on IRT and raw score perform somewhat better at longer test lengths. Although these differences between methods are detectable, they are small in magnitude.