Cascaded one-shot deformable convolutional neural networks: Developing a deep learning model for respiratory motion estimation in ultrasound sequences

• Published in: Medical image analysis Vol. 65; p. 101793
• Main Authors: Liu, Fei, Liu, Dan, Tian, Jie, Xie, Xiaoyan, Yang, Xin, Wang, Kun
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.10.2020; Elsevier BV
• Subjects: Artificial neural networks; Cascaded Siamese network; Convolution; Datasets; Deep learning; Formability; Image quality; Medical imaging; Motion simulation; Neural networks; One-shot deformable convolution; Radiation; Radiation therapy; Respiratory motion estimation; Robustness; Signal to noise ratio; Tracking; Ultrasonic imaging; Ultrasound; Ultrasound sequence; Ultrasound sequence; Cascaded Siamese network; Respiratory motion estimation; One-shot deformable convolution
• ISSN: 1361-8415; 1361-8423; 1361-8423
• DOI: 10.1016/j.media.2020.101793

•COSD-CNN is a cascaded Siamese network with one-shot deformable convolution module.•COSD-CNN alleviates performance degradation from network subsampling.•COSD-CNN is robust to target appearance changes across ultrasound images.•COSD-CNN achieves state-of-the-art performance on the CLUST benchmark dataset. Improving the quality of image-guided radiation therapy requires the tracking of respiratory motion in ultrasound sequences. However, the low signal-to-noise ratio and the artifacts in ultrasound images make it difficult to track targets accurately and robustly. In this study, we propose a novel deep learning model, called a Cascaded One-shot Deformable Convolutional Neural Network (COSD-CNN), to track landmarks in real time in long ultrasound sequences. Specifically, we design a cascaded Siamese network structure to improve the tracking performance of CNN-based methods. We propose a one-shot deformable convolution module to enhance the robustness of the COSD-CNN to appearance variation in a meta-learning manner. Moreover, we design a simple and efficient unsupervised strategy to facilitate the network's training with a limited number of medical images, in which many corner points are selected from raw ultrasound images to learn network features with high generalizability. The proposed COSD-CNN has been extensively evaluated on the public Challenge on Liver UltraSound Tracking (CLUST) 2D dataset and on our own ultrasound image dataset from the First Affiliated Hospital of Sun Yat-sen University (FSYSU). Experiment results show that the proposed model can track a target through an ultrasound sequence with high accuracy and robustness. Our method achieves new state-of-the-art performance on the CLUST 2D benchmark set, indicating its strong potential for application in clinical practice. [Display omitted]