Using generalizability theory with continuous latent response variables

• Published in: Psychological methods Vol. 24; no. 2; p. 153
• Main Authors: Vispoel, Walter P, Morris, Carrie A, Kilinc, Murat
• Format: Journal Article
• Language: English
• Published: United States 01.04.2019
• Subjects: Data Interpretation, Statistical; Humans; Models, Statistical; Psychology - methods; Reproducibility of Results; Research Design
• ISSN: 1939-1463
• DOI: 10.1037/met0000177

In this article, we illustrate ways in which generalizability theory (G-theory) can be used with continuous latent response variables (CLRVs) to address problems of scale coarseness resulting from categorization errors caused by representing ranges of continuous variables by discrete data points and transformation errors caused by unequal interval widths between those data points. The mechanism to address these problems is applying structural equation modeling (SEM) as a tool in deriving variance components needed to estimate indices of score consistency and validity. Illustrations include quantification of multiple sources of measurement error, use of non-nested and nested designs, derivation of indices of consistency for norm- and criterion-referenced interpretation of scores, estimation of effects when changing measurement procedures and designs, and disattenuation of correlation coefficients for measurement error. These illustrations underscore the effectiveness of G-theory with continuous latent response variables in providing stable indices of reliability and validity that are reasonably independent of the number of original scale points used, unevenness of scale intervals, and average degree of item skewness. We discuss general distinctions in reliability estimation within G-theory, SEM, and classical test theory; make specific recommendations for using G-theory on raw score and CLRV metrics; and provide computer code in an online supplement for doing all key analyses demonstrated in the article using R and M (PsycINFO Database Record (c) 2019 APA, all rights reserved).