Self-supervised learning for medical image analysis using image context restoration

• Published in: Medical image analysis Vol. 58; p. 101539
• Main Authors: Chen, Liang, Bentley, Paul, Mori, Kensaku, Misawa, Kazunari, Fujiwara, Michitaka, Rueckert, Daniel
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01.12.2019; Elsevier BV
• Subjects: Abdomen - diagnostic imaging; Brain cancer; Brain Neoplasms - diagnostic imaging; Brain tumors; Classification; Computed tomography; Context restoration; Deep learning; Fetuses; Humans; Image analysis; Image detection; Image processing; Image segmentation; Learning algorithms; Localization; Machine learning; Magnetic Resonance Imaging; Medical image analysis; Medical imaging; Neuroimaging; Organs; Restoration strategies; Self-supervised learning; Semantics; Supervised Machine Learning; Tomography, X-Ray Computed; Training; Ultrasonography, Prenatal; Ultrasound; Self-supervised learning; Context restoration; Medical image analysis
• ISSN: 1361-8415; 1361-8423; 1361-8423
• DOI: 10.1016/j.media.2019.101539

•A novel self-supervised learning strategy called context restoration.•It improves the subsequent learning performance.•Its implementation is simple and straightforward.•It is useful for different types of subsequent tasks, including classification, detection, and segmentation. [Display omitted] Machine learning, particularly deep learning has boosted medical image analysis over the past years. Training a good model based on deep learning requires large amount of labelled data. However, it is often difficult to obtain a sufficient number of labelled images for training. In many scenarios the dataset in question consists of more unlabelled images than labelled ones. Therefore, boosting the performance of machine learning models by using unlabelled as well as labelled data is an important but challenging problem. Self-supervised learning presents one possible solution to this problem. However, existing self-supervised learning strategies applicable to medical images cannot result in significant performance improvement. Therefore, they often lead to only marginal improvements. In this paper, we propose a novel self-supervised learning strategy based on context restoration in order to better exploit unlabelled images. The context restoration strategy has three major features: 1) it learns semantic image features; 2) these image features are useful for different types of subsequent image analysis tasks; and 3) its implementation is simple. We validate the context restoration strategy in three common problems in medical imaging: classification, localization, and segmentation. For classification, we apply and test it to scan plane detection in fetal 2D ultrasound images; to localise abdominal organs in CT images; and to segment brain tumours in multi-modal MR images. In all three cases, self-supervised learning based on context restoration learns useful semantic features and lead to improved machine learning models for the above tasks.