Natural frequency tree- versus conditional probability formula-based training for medical students' estimation of screening test predictive values: a randomized controlled trial

• Published in: BMC medical education Vol. 24; no. 1; pp. 1207 - 12
• Main Authors: Kim, Soela, Kim, Soyun, Choi, Yong-Jun, Do, Young Kyung
• Format: Journal Article
• Language: English
• Published: England BioMed Central Ltd 24.10.2024; BioMed Central; BMC
• Subjects: Accuracy; Adult; Calculators; Clinical Competence; Clinical trials; Conditional probability; COVID-19; Curricula; Disease; Education, Medical, Undergraduate - methods; Educational films; Educational Measurement; Female; Home medical tests; Humans; Inferences; Information services; Information services industry; Laboratories; Male; Mathematical Formulas; Mathematics Education; Medical colleges; Medical Education; Medical personnel; Medical research; Medical schools; Medical screening; Medical Services; Medical students; Medical tests; Medicine, Experimental; Natural frequency; Pandemics; Predictive value of screening tests; Predictive Value of Tests; Probabilistic reasoning; Probability; Questionnaires; Randomized Controlled Trials; Republic of Korea; Screening Tests; Statistical Inference; Student Surveys; Students, Medical; Test Results; Training; Training Methods; Video Technology; Visualization; Young Adult; Republic of Korea; Canada; South Korea; Medical students; Natural frequency; Conditional probability; Probabilistic reasoning; Predictive value of screening tests
• ISSN: 1472-6920; 1472-6920
• DOI: 10.1186/s12909-024-06209-0

Medical students and professionals often struggle to understand medical test results, which can lead to poor medical decisions. Natural frequency tree-based training (NF-TT) has been suggested to help people correctly estimate the predictive value of medical tests. We aimed to compare the effectiveness of NF-TT with conventional conditional probability formula-based training (CP-FT) and investigate student variables that may influence NF-TT's effectiveness. We conducted a parallel group randomized controlled trial of NF-TT vs. CP-FT in two medical schools in South Korea (a 1:1 allocation ratio). Participants were randomly assigned to watch either NF-TT or CP-FT video at individual computer stations. NF-TT video showed how to translate relevant probabilistic information into natural frequencies using a tree structure to estimate the predictive values of screening tests. CP-FT video showed how to plug the same information into a mathematical formula to calculate predictive values. Both videos were 15 min long. The primary outcome was the accuracy in estimating the predictive value of screening tests assessed using multiple-choice questions at baseline, post-intervention (i.e., immediately after training), and one-month follow-up. The secondary outcome was the accuracy of conditional probabilistic reasoning in non-medical contexts, also assessed using multiple-choice questions, but only at follow-up as a measure of transfer of learning. 231 medical students completed their participation. Overall, NF-TT was not more effective than CP-FT in improving the predictive value estimation accuracy at post-intervention (NF-TT: 87.13%, CP-FT: 86.03%, p = .86) and follow-up (NF-TT: 72.39%, CP-FT: 68.10%, p = .40) and facilitating transfer of training (NF-TT: 75.54%, CP-FT: 71.43%, p = .41). However, for participants without relevant prior training, NF-TT was more effective than CP-FT in improving estimation accuracy at follow-up (NF-TT: 74.86%, CP-FT: 58.71%, p = .02) and facilitating transfer of learning (NF-TT: 82.86%, CP-FT: 66.13%, p = .04). Introducing NF-TT early in the medical school curriculum, before students are exposed to a pervasive conditional probability formula-based approach, would offer the greatest benefit. Korea Disease Control and Prevention Agency Clinical Research Information Service KCT0004246 (the date of first trial registration: 27/08/2019). The full trial protocol can be accessed at https://cris.nih.go.kr/cris/search/detailSearch.do?seq=15616&search_page=L .