Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches

• Published in: Behavior research methods Vol. 41; no. 2; pp. 372 - 384
• Main Authors: Ferron, John M., Bell, Bethany A., Hess, Melinda R., Rendina-Gobioff, Gianna, Hibbard, Susan T.
• Format: Journal Article
• Language: English
• Published: New York Springer-Verlag 01.05.2009; Psychonomic Society, Inc
• Subjects: Algorithms; Autism; Behavioral Research - statistics & numerical data; Behavioral Science and Psychology; Cognitive Psychology; Data Interpretation, Statistical; Estimates; Humans; Intellectual disabilities; Methods; Models, Statistical; Monte Carlo Method; Psychology; Psychomotor Performance - physiology; Reference Values; Social interaction; Studies; FERRON; Interval Estimate; Average Treatment Effect; Autocorrelation; Error Structure
• ISSN: 1554-351X; 1554-3528
• DOI: 10.3758/BRM.41.2.372

Multiple-baseline studies are prevalent in behavioral research, but questions remain about how to best analyze the resulting data. Monte Carlo methods were used to examine the utility of multilevel models for multiplebaseline data under conditions that varied in the number of participants, number of repeated observations per participant, variance in baseline levels, variance in treatment effects, and amount of autocorrelation in the Level 1 errors. Interval estimates of the average treatment effect were examined for two specifications of the Level 1 error structure (σ 2 I and first-order autoregressive) and for five different methods of estimating the degrees of freedom (containment, residual, between—within, Satterthwaite, and Kenward—Roger). When the Satterthwaite or Kenward—Roger method was used and an autoregressive Level 1 error structure was specified, the interval estimates of the average treatment effect were relatively accurate. Conversely, the interval estimates of the treatment effect variance were inaccurate, and the corresponding point estimates were biased.