Semi-supervised learning for automatic segmentation of the knee from MRI with convolutional neural networks

• Published in: Computer methods and programs in biomedicine Vol. 189; p. 105328
• Main Authors: Burton, William, Myers, Casey, Rullkoetter, Paul
• Format: Journal Article
• Language: English
• Published: Ireland Elsevier B.V 01.06.2020
• Subjects: Algorithms; Convolutional neural networks (CNNs); Deep Learning; Humans; Image Processing, Computer-Assisted - methods; Imaging, Three-Dimensional; Knee; Knee Joint - diagnostic imaging; Magnetic Resonance Imaging; Medical Imaging; Monte Carlo Method; Monte Carlo patch sampling; Neural Networks, Computer; Segmentation; Semi-supervised; Supervised Machine Learning; Knee; Segmentation; Magnetic resonance imaging; Monte Carlo patch sampling; Semi-supervised; Convolutional neural networks (CNNs); Medical Imaging
• ISSN: 0169-2607; 1872-7565; 1872-7565
• DOI: 10.1016/j.cmpb.2020.105328

•Convolutional neural networks were developed for segmentation of the knee from MRI.•Semi-supervised methods leveraged unlabeled data for improved performance.•Predicted segmentation maps were used to develop finite element models.•CNN performance was competitive despite relying on small amounts of labeled data. Segmentation is a crucial step in multiple biomechanics and orthopedics applications. The time-intensiveness and expertise requirements of medical image segmentation present a significant bottleneck for corresponding workflows. The current study develops and evaluates convolutional neural networks (CNNs) for automatic segmentation of magnetic resonance imaging (MRI) with the objective of assessing their utility for use in biomechanics research methods. CNNs were developed using a previously published, fully-annotated dataset as well as unlabeled scans from a publicly-available dataset. 2D and 3D CNNs were trained using semi-supervised learning frameworks for automatic segmentation of six structures of the knee. An inference strategy called Monte Carlo patch sampling was introduced to increase accuracy of the resulting models while adding no additional steps to the training process. Performance was assessed using traditional segmentation metrics, as well as surface error between reconstructed geometries from predicted and manual segmentations. Geometries from predicted segmentation maps were developed into finite element (FE) models in a semi-automatic pipeline and evaluated for FE-readiness. 3D CNNs using Monte Carlo patch sampling during inference achieved an Intersection-over-Union (IoU) of 0.978 and a dice similarity coefficient (DSC) of 0.989. Median surface error between predicted and ground truth geometries ranged from 0.56 to 0.98 mm. Meshes generated from the predicted segmentation maps were successfully used in FE simulations, demonstrating FE-readiness of geometries predicted by CNNs. CNNs trained with semi-supervised techniques outperformed CNNs trained in a fully-supervised fashion and resulted in performance competitive with similar literature despite relying on significantly less labeled data. CNNs developed for automatic segmentation have potential for supplementing manual segmentation workflows in a wide range of orthopedics and biomechanics applications, including FE analysis. Faster processing times for developing FE models can enable population-based FE analysis using subject-specific models. The use of semi-supervised learning algorithms may additionally help circumvent the cost of obtaining labeled data in the development of these models.