Methods used to assess outcome consistency in clinical studies: A literature-based evaluation

• Published in: PloS one Vol. 15; no. 7; p. e0235485
• Main Authors: Rogozinska, Ewelina, Gargon, Elizabeth, Olmedo-Requena, Rocío, Asour, Amani, Cooper, Natalie A. M, Vale, Claire L, van't Hooft, Janneke, Mathes, Tim
• Format: Journal Article
• Language: English
• Published: San Francisco Public Library of Science 08.07.2020; Public Library of Science (PLoS)
• Subjects: Clinical trials; Consistency; Data bases; Data collection; Domains; Evaluation; Handbooks; Health care; Identification; Labeling; Labelling; Literature reviews; Medical research; Medicine and Health Sciences; Methods; Outcome and process assessment (Medical care); Patient outcomes; Preventive medicine; Public health; Research and Analysis Methods; Research design; Reviews; Studies; Systematic review; United Kingdom; United Kingdom > UK; England; Spain
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0235485

Evaluation studies of outcomes used in clinical research and their consistency are appearing more frequently in the literature, as a key part of the core outcome set (COS) development. Current guidance suggests such evaluation studies should use systematic review methodology as their default. We aimed to examine the methods used. We searched the Core Outcome Measures in Effectiveness Trials (COMET) database (up to May 2019) supplementing it with additional resources. We included evaluation studies of outcome consistency in clinical studies across health subjects and used a subset of A MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2 (items 1-9) to assess their methods. Of 93 included evaluation studies of outcome consistency (90 full reports, three summaries), 91% (85/93) reported performing literature searches in at least one bibliographic database, and 79% (73/93) was labelled as a "systematic review". The evaluations varied in terms of satisfying AMSTAR 2 criteria, such that 81/93 (87%) had implemented PICO in the research question, whereas only 5/93 (6%) had included the exclusions list. None of the evaluation studies explained how inconsistency of outcomes was detected, however, 80/90 (88%) concluded inconsistency in individual outcomes (66%, 55/90) or outcome domains (20%, 18/90). Methods used in evaluation studies of outcome consistency in clinical studies differed considerably. Despite frequent being labelled as a "systematic review", adoption of systematic review methodology is selective. While the impact on COS development is unknown, authors of these studies should refrain from labelling them as "systematic review" and focus on ensuring that the methods used to generate the different outcomes and outcome domains are reported transparently.