Multitask Cascade Convolution Neural Networks for Automatic Thyroid Nodule Detection and Recognition

• Published in: IEEE journal of biomedical and health informatics Vol. 23; no. 3; pp. 1215 - 1224
• Main Authors: Song, Wenfeng, Li, Shuai, Liu, Ji, Qin, Hong, Zhang, Bo, Zhang, Shuyang, Hao, Aimin
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.05.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Cancer; Case studies; Convolution; detection; Diagnosis; Feature extraction; Ground truth; Humans; Image Interpretation, Computer-Assisted - methods; Image recognition; Learning algorithms; Machine learning; Medical personnel; Neural networks; Neural Networks, Computer; Nodules; Object recognition; Predictive models; pyramid convolution neural networks; recognition; Shape; Therapeutic applications; Thyroid; Thyroid gland; Thyroid Gland - diagnostic imaging; Thyroid Nodule - diagnostic imaging; Thyroid nodules; Ultrasonic imaging; Ultrasonography; Ultrasound
• ISSN: 2168-2194; 2168-2208; 2168-2208
• DOI: 10.1109/JBHI.2018.2852718

Thyroid ultrasonography is a widely used clinical technique for nodule diagnosis in thyroid regions. However, it remains difficult to detect and recognize the nodules due to low contrast, high noise, and diverse appearance of nodules. In today's clinical practice, senior doctors could pinpoint nodules by analyzing global context features, local geometry structure, and intensity changes, which would require rich clinical experience accumulated from hundreds and thousands of nodule case studies. To alleviate doctors' tremendous labor in the diagnosis procedure, we advocate a machine learning approach to the detection and recognition tasks in this paper. In particular, we develop a multitask cascade convolution neural network (MC-CNN) framework to exploit the context information of thyroid nodules. It may be noted that our framework is built upon a large number of clinically confirmed thyroid ultrasound images with accurate and detailed ground truth labels. Other key advantages of our framework result from a multitask cascade architecture, two stages of carefully designed deep convolution networks in order to detect and recognize thyroid nodules in a pyramidal fashion, and capturing various intrinsic features in a global-to-local way. Within our framework, the potential regions of interest after initial detection are further fed to the spatial pyramid augmented CNNs to embed multiscale discriminative information for fine-grained thyroid recognition. Experimental results on 4309 clinical ultrasound images have indicated that our MC-CNN is accurate and effective for both thyroid nodules detection and recognition. For the correct diagnosis rate of malignant and benign thyroid nodules, its mean Average Precision (mAP) performance can achieve up to <inline-formula><tex-math notation="LaTeX">\text{98.2}\%</tex-math></inline-formula> accuracy, which outperforms the common CNNs by <inline-formula><tex-math notation="LaTeX">\text{5}\%</tex-math></inline-formula> on average. In addition, we conduct rigorous user studies to confirm that our MC-CNN outperforms experienced doctors, yet only consuming roughly <inline-formula><tex-math notation="LaTeX">\text{2}\%</tex-math></inline-formula> (<inline-formula><tex-math notation="LaTeX">1/48</tex-math></inline-formula>) of doctors' examination time on average. Therefore, the accuracy and efficiency of our new method exhibit its great potential in clinical applications.