Bi-Directional Semi-Supervised Training of Convolutional Neural Networks for Ultrasound Elastography Displacement Estimation

• Published in: IEEE transactions on ultrasonics, ferroelectrics, and frequency control Vol. 69; no. 4; pp. 1181 - 1190
• Main Authors: Tehrani, Ali K. Z., Sharifzadeh, Mostafa, Boctor, Emad, Rivaz, Hassan
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.04.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Computer vision; Convolutional neural network (CNN); Costs; Deep learning; Displacement; displacement estimation; Elasticity Imaging Techniques - methods; Estimation; In vivo methods and tests; Machine learning; Neural networks; Neural Networks, Computer; Optical flow; Optical flow (image analysis); Optical losses; Optimization; Performance enhancement; Phantoms, Imaging; Radio frequency; Regularization; semisupervised training; Training; Ultrasonic imaging; Ultrasonography; ultrasound elastography (USE)
• ISSN: 0885-3010; 1525-8955
• DOI: 10.1109/TUFFC.2022.3147097

The performance of ultrasound elastography (USE) heavily depends on the accuracy of displacement estimation. Recently, convolutional neural networks (CNNs) have shown promising performance in optical flow estimation and have been adopted for USE displacement estimation. Networks trained on computer vision images are not optimized for USE displacement estimation since there is a large gap between the computer vision images and the high-frequency radio frequency (RF) ultrasound data. Many researchers tried to adopt the optical flow CNNs to USE by applying transfer learning to improve the performance of CNNs for USE. However, the ground-truth displacement in real ultrasound data is unknown, and simulated data exhibit a domain shift compared to the real data and are also computationally expensive to generate. To resolve this issue, semisupervised methods have been proposed in which the networks pretrained on computer vision images are fine-tuned using real ultrasound data. In this article, we employ a semisupervised method by exploiting the first- and second-order derivatives of the displacement field for regularization. We also modify the network structure to estimate both forward and backward displacements and propose to use consistency between the forward and backward strains as an additional regularizer to further enhance the performance. We validate our method using several experimental phantom and in vivo data. We also show that the network fine-tuned by our proposed method using experimental phantom data performs well on in vivo data similar to the network fine-tuned on in vivo data. Our results also show that the proposed method outperforms current deep learning methods and is comparable to computationally expensive optimization-based algorithms.