Real needle for minimal invasive procedures training using motion sensors and optical flow

• Published in: Computers in biology and medicine Vol. 170; p. 107935
• Main Authors: García-Esteban, J.A., Curto, B., Moreno, V., Hernández, F., Alonso, P., Serrano, F.J., Blanco, F.J.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.03.2024; Elsevier Limited
• Subjects: Accuracy; Algorithms; Attitudes; Biology; Cameras; Computer Simulation; Computers; Embedded systems; Embedded vision; Estimation; Haptic; Haptics; Humans; Inertial platforms; Inertial sensing devices; Insertion; Localization; Medical personnel; Medical simulator training; Medicine; Motion sensors; Motor skill; Movement; Needles; Optic Flow; Optical flow (image analysis); Sensorial fusion; Sensors; Simulation; Skills; Success; Training; Training simulators; User-Computer Interface; Vision systems; Sensorial fusion; Embedded systems; Medical simulator training; Embedded vision; Haptic
• ISSN: 0010-4825; 1879-0534
• DOI: 10.1016/j.compbiomed.2024.107935

Minimally invasive percutaneous insertion procedures are widely used techniques in medicine. Their success is highly dependent on the skills of the practitioner. This paper presents a haptic simulator for training in these procedures, whose key component is a real percutaneous insertion needle with a sensory system incorporated to track its 3D location at every instant. By means of the proposed embedded vision system, the attitude (spatial orientation) and depth of insertion of a real needle are estimated. The proposal is founded on a novel depth estimation procedure based on optical flow techniques, complemented by sensory fusion techniques with the attitude calculated with data from an Inertial Measurement Unit (IMU) sensor. This procedure allows estimating the needle attitude with an accuracy of tenths of a degree and the displacement with an accuracy of millimeters. The computational algorithm runs on an embedded computer with real-time constraints for tracking the movement of a real needle. This haptic needle location data is used to reproduce the movement of a virtual needle within a simulation app. As a fundamental result, an ergonomic and realistic training simulator has been successfully constructed for healthcare professionals to acquire the mental model and motor skills necessary to practice percutaneous procedures successfully. •Needle real-time motion tracking for realistic percutaneous simulations.•A powerful sensor fusion off-the-shelf MEMS & vision devices in embedded hardware.•Needle travel measures based on a new optical flow approach with sparse fact images.•Realistic medical training simulator for skill development at percutaneous insertion.•Powerful real-time needle motion rendering application for optimal student learning.