The use of generalizability theory to estimate data reliability in single-subject observational research

• Published in: Psychology in the schools Vol. 44; no. 5; pp. 433 - 439
• Main Authors: Lei, Pui-Wa, Smith, Maria, Suen, Hoi K.
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc., A Wiley Company 01.05.2007; John Wiley & Sons, Inc; Wiley
• Subjects: Biological and medical sciences; Data Collection; Educational psychology; Evaluation Methods; Fundamental and applied biological sciences. Psychology; Generalizability; Generalizability Theory; Observation; Observational research; Orientation. Evaluation; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Reliability; Human; Methodology; Theory; Professional practice; Observation; Classroom; Psychometrics; School psychology; Test reliability; Test administration
• ISSN: 0033-3085; 1520-6807
• DOI: 10.1002/pits.20235

Direct observation of behaviors is a data collection method customarily used in clinical and educational settings. Repeated measures and small samples are inherent characteristics of observational studies that pose challenges to the numerical estimation of reliability for observational data. In this article, we review some debates about the use of Generalizability Theory in estimating reliability of single‐subject observational data. We propose that it could be used but under a clearly stated set of conditions. The conceptualization of facets and object of measurement for a common design of observational research is elucidated under a different light. We provide two numerical examples to illustrate the ideas. Limitations of using Generalizability Theory to estimate reliability of observational data are discussed. © 2007 Wiley Periodicals, Inc. Psychol Schs 44: 433–439, 2007.