Revision of a method quality rating scale for single-case experimental designs and n-of-1 trials: The 15-item Risk of Bias in N-of-1 Trials (RoBiNT) Scale

• Published in: Neuropsychological rehabilitation Vol. 23; no. 5; pp. 619 - 638
• Main Authors: Tate, Robyn L, Perdices, Michael, Rosenkoetter, Ulrike, Wakim, Donna, Godbee, Kali, Togher, Leanne, McDonald, Skye
• Format: Journal Article
• Language: English
• Published: Hove Routledge 01.10.2013; Psychology Press; Taylor & Francis Ltd
• Subjects: Bias; Biological and medical sciences; Data Interpretation, Statistical; Evidence-based medicine; Humans; Internal validity; Medical sciences; Methodology; Miscellaneous; n-of-1 trial; Novices; Psychology. Psychoanalysis. Psychiatry; Psychometric properties; Psychometrics - methods; Psychopathology. Psychiatry; Randomized Controlled Trials as Topic - methods; Ratings & rankings; Research Design; Revisions; Revival; Risk of bias; Single-case experimental design; Techniques and methods; Validity; Human; Methodology; Evaluation scale; M-of-1 trial; Evidence-based practice; Internal validity; Single-case experimental design; Experimental study; Evidence-based medicine; Case study; Validity; Treatment; Risk of bias; Experimental design; Quality; Risk factor; Methodological bias
• ISSN: 0960-2011; 1464-0694; 1464-0694
• DOI: 10.1080/09602011.2013.824383

Recent literature suggests a revival of interest in single-case methodology (e.g., the randomised n-of-1 trial is now considered Level 1 evidence for treatment decision purposes by the Oxford Centre for Evidence-Based Medicine). Consequently, the availability of tools to critically appraise single-case reports is of great importance. We report on a major revision of our method quality instrument, the Single-Case Experimental Design Scale. Three changes resulted in a radically revised instrument, now entitled the Risk of Bias in N-of-1 Trials (RoBiNT) Scale: (i) item content was revised and increased to 15 items, (ii) two subscales were developed for internal validity (IV; 7 items) and external validity and interpretation (EVI; 8 items), and (iii) the scoring system was changed from a 2-point to 3-point scale to accommodate currently accepted standards. Psychometric evaluation indicated that the RoBiNT Scale showed evidence of construct (discriminative) validity. Inter-rater reliability was excellent, for pairs of both experienced and trained novice raters. Intraclass correlation coefficients of summary scores for individual (experienced) raters: ICC TotalScore  = .90, ICC IVSubscale  = .88, ICC EVISubscale  = .87; individual (novice) raters: ICC TotalScore  = .88, ICC IVSubscale  = .87, ICC EVISubscale  = .93; consensus ratings between experienced and novice raters (ICC TotalScore  = .95, ICC IVSubscale  = .93, ICC EVISubscale  = .93. The RoBiNT Scale thus shows sound psychometric properties and provides a comprehensive yet efficient examination of important features of single-case methodology.