Automatic needle tracking using Mask R-CNN for MRI-guided percutaneous interventions

• Published in: International journal for computer assisted radiology and surgery Vol. 15; no. 10; pp. 1673 - 1684
• Main Authors: Li, Xinzhou, Young, Adam S., Raman, Steven S., Lu, David S., Lee, Yu-Hsiu, Tsao, Tsu-Chin, Wu, Holden H.
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.10.2020; Springer Nature B.V
• Subjects: Algorithms; Artificial neural networks; Biopsy; Computer Imaging; Computer Science; Datasets; Euclidean geometry; Feature extraction; Health Informatics; Humans; Image segmentation; Image-Guided Biopsy - methods; Imaging; Magnetic resonance imaging; Magnetic Resonance Imaging - methods; Male; Medicine; Medicine & Public Health; Needles; Neural Networks, Computer; Original Article; Pattern Recognition and Graphics; Percutaneous treatment; Prostate; Prostate - pathology; Radiology; Real time; Surgery; Tracking; Vision; Deep learning; Needle feature; Interventional MRI; Device tracking; Convolutional neural network; Real-time MRI
• ISSN: 1861-6410; 1861-6429; 1861-6429
• DOI: 10.1007/s11548-020-02226-8

Purpose Accurate needle tracking provides essential information for MRI-guided percutaneous interventions. Passive needle tracking using MR images is challenged by variations of the needle-induced signal void feature in different situations. This work aimed to develop an automatic needle tracking algorithm for MRI-guided interventions based on the Mask Region Proposal-Based Convolutional Neural Network (R-CNN). Methods Mask R-CNN was adapted and trained to segment the needle feature using 250 intra-procedural images from 85 MRI-guided prostate biopsy cases and 180 real-time images from MRI-guided needle insertion in ex vivo tissue. The segmentation masks were passed into the needle feature localization algorithm to extract the needle feature tip location and axis orientation. The proposed algorithm was tested using 208 intra-procedural images from 40 MRI-guided prostate biopsy cases, and 3 real-time MRI datasets in ex vivo tissue. The algorithm results were compared with human-annotated references. Results In prostate datasets, the proposed algorithm achieved needle feature tip localization error with median Euclidean distance (d xy ) of 0.71 mm and median difference in axis orientation angle (d θ ) of 1.28°, respectively. In 3 real-time MRI datasets, the proposed algorithm achieved consistent dynamic needle feature tracking performance with processing time of 75 ms/image: (a) median d xy  = 0.90 mm, median d θ  = 1.53°; (b) median d xy  = 1.31 mm, median d θ  = 1.9°; (c) median d xy  = 1.09 mm, median d θ  = 0.91°. Conclusions The proposed algorithm using Mask R-CNN can accurately track the needle feature tip and axis on MR images from in vivo intra-procedural prostate biopsy cases and ex vivo real-time MRI experiments with a range of different conditions. The algorithm achieved pixel-level tracking accuracy in real time and has potential to assist MRI-guided percutaneous interventions.