Assessing colonoscopic inspection skill using a virtual withdrawal simulation: a preliminary validation of performance metrics

• Published in: BMC medical education Vol. 17; no. 1; p. 118
• Main Authors: Zupanc, Christine M, Wallis, Guy M, Hill, Andrew, Burgess-Limerick, Robin, Riek, Stephan, Plooy, Annaliese M, Horswill, Mark S, Watson, Marcus O, de Visser, Hans, Conlan, David, Hewett, David G
• Format: Journal Article
• Language: English
• Published: England BioMed Central Ltd 12.07.2017; BioMed Central; BMC
• Subjects: Adult; Analysis; Australia; Business metrics; Cancer; Clinical Competence; Colonic Neoplasms - pathology; Colonoscopy; Colonoscopy - education; Colonoscopy - standards; Colorectal cancer; Comparative analysis; Computer Simulation; Construct Validity; Educational Measurement; Effect Size; Endoscopy; Female; Humans; Inspection; Male; Medical personnel; Medical Students; Middle Aged; Models, Educational; Novices; Performance-based assessment; Reproducibility of Results; Simulation; Simulation methods; Skill assessment; Students, Medical; Task analysis; Task Performance and Analysis; Training; Tumors; Validity; Virtual reality; Young Adult; Training; Virtual reality; Skill assessment
• ISSN: 1472-6920; 1472-6920
• DOI: 10.1186/s12909-017-0948-6

The effectiveness of colonoscopy for diagnosing and preventing colon cancer is largely dependent on the ability of endoscopists to fully inspect the colonic mucosa, which they achieve primarily through skilled manipulation of the colonoscope during withdrawal. Performance assessment during live procedures is problematic. However, a virtual withdrawal simulation can help identify and parameterise actions linked to successful inspection, and offer standardised assessments for trainees. Eleven experienced endoscopists and 18 endoscopy novices (medical students) completed a mucosal inspection task during three simulated colonoscopic withdrawals. The two groups were compared on 10 performance metrics to preliminarily assess the validity of these measures to describe inspection quality. Four metrics were related to aspects of polyp detection: percentage of polyp markers found; number of polyp markers found per minute; percentage of the mucosal surface illuminated by the colonoscope (≥0.5 s); and percentage of polyp markers illuminated (≥2.5 s) but not identified. A further six metrics described the movement of the colonoscope: withdrawal time; linear distance travelled by the colonoscope tip; total distance travelled by the colonoscope tip; and distance travelled by the colonoscope tip due to movement of the up/down angulation control, movement of the left/right angulation control, and axial shaft rotation. Statistically significant experienced-novice differences were found for 8 of the 10 performance metrics (p's < .005). Compared with novices, experienced endoscopists inspected more of the mucosa and detected more polyp markers, at a faster rate. Despite completing the withdrawals more quickly than the novices, the experienced endoscopists also moved the colonoscope more in terms of linear distance travelled and overall tip movement, with greater use of both the up/down angulation control and axial shaft rotation. However, the groups did not differ in the number of polyp markers visible on the monitor but not identified, or movement of the left/right angulation control. All metrics that yielded significant group differences had adequate to excellent internal consistency reliability (α = .79 to .90). These systematic differences confirm the potential of the simulated withdrawal task for evaluating inspection skills and strategies. It may be useful for training, and assessment of trainee competence.