Using a large language model (ChatGPT) to assess risk of bias in randomized controlled trials of medical interventions: protocol for a pilot study of interrater agreement with human reviewers

• Published in: BMC medical research methodology Vol. 25; no. 1; pp. 182 - 11
• Main Authors: Rose, Christopher James, Bidonde, Julia, Ringsten, Martin, Glanville, Julie, Berg, Rigmor C., Cooper, Chris, Muller, Ashley Elizabeth, Bergsund, Hans Bugge, Meneses-Echavez, Jose F., Potrebny, Thomas
• Format: Journal Article
• Language: English
• Published: England BioMed Central Ltd 31.07.2025; BioMed Central; BMC
• Subjects: Agreements; Annan klinisk medicin; Artificial intelligence; Automation; Bias; Chatbots; ChatGPT; Clinical Medicine; Clinical trials; Dexmedetomidine; Evaluation; Generative Artificial Intelligence; Homeopathy; Humans; Klinisk medicin; Language; Large language model; Large Language Models; Machine Learning; Materia medica and therapeutics; Medical and Health Sciences; Medicin och hälsovetenskap; Observer Variation; Other Clinical Medicine; Pilot Projects; Randomized Controlled Trials as Topic - methods; Randomized Controlled Trials as Topic - statistics & numerical data; Research Design; Risk Assessment - methods; Risk of bias; Set (Psychology); Systematic reviewing; Systematic Reviews as Topic; Therapeutics; Therapeutics, Experimental; Norway; United States > US; Large language model; Risk of bias; Systematic reviewing; ChatGPT; Artificial intelligence; Machine learning

Risk of bias (RoB) assessment is an essential part of systematic reviews that requires reading and understanding each eligible trial and RoB tools. RoB assessment is subject to human error and is time-consuming. Machine learning-based tools have been developed to automate RoB assessment using simple...