NiftyNet: a deep-learning platform for medical imaging

• Published in: Computer methods and programs in biomedicine Vol. 158; pp. 113 - 122
• Main Authors: Gibson, Eli, Li, Wenqi, Sudre, Carole, Fidon, Lucas, Shakir, Dzhoshkun I., Wang, Guotai, Eaton-Rosen, Zach, Gray, Robert, Doel, Tom, Hu, Yipeng, Whyntie, Tom, Nachev, Parashkev, Modat, Marc, Barratt, Dean C., Ourselin, Sébastien, Cardoso, M. Jorge, Vercauteren, Tom
• Format: Journal Article
• Language: English
• Published: Ireland Elsevier B.V 01.05.2018; Elsevier Scientific Publishers
• Subjects: Abdomen - diagnostic imaging; Brain - diagnostic imaging; Computer Simulation; Convolutional neural network; Databases, Factual; Deep learning; Diagnostic Imaging - instrumentation; Diagnostic Imaging - methods; Generative adversarial network; Humans; Image Processing, Computer-Assisted - instrumentation; Image Processing, Computer-Assisted - methods; Image regression; Machine Learning; Magnetic Resonance Imaging; Medical image analysis; Neural Networks, Computer; Segmentation; Ultrasonography; Deep learning; Segmentation; Convolutional neural network; Medical image analysis; Generative adversarial network; Image regression
• ISSN: 0169-2607; 1872-7565; 1872-7565
• DOI: 10.1016/j.cmpb.2018.01.025

•An open-source platform is implemented based on TensorFlow APIs for deep learning in medical imaging domain.•A modular implementation of the typical medical imaging machine learning pipeline facilitates (1) warm starts with established pre-trained networks, (2) adapting existing neural network architectures to new problems, and (3) rapid prototyping of new solutions.•Three deep-learning applications, including segmentation, regression, image generation and representation learning, are presented as concrete examples illustrating the platform’s key features. Medical image analysis and computer-assisted intervention problems are increasingly being addressed with deep-learning-based solutions. Established deep-learning platforms are flexible but do not provide specific functionality for medical image analysis and adapting them for this domain of application requires substantial implementation effort. Consequently, there has been substantial duplication of effort and incompatible infrastructure developed across many research groups. This work presents the open-source NiftyNet platform for deep learning in medical imaging. The ambition of NiftyNet is to accelerate and simplify the development of these solutions, and to provide a common mechanism for disseminating research outputs for the community to use, adapt and build upon. The NiftyNet infrastructure provides a modular deep-learning pipeline for a range of medical imaging applications including segmentation, regression, image generation and representation learning applications. Components of the NiftyNet pipeline including data loading, data augmentation, network architectures, loss functions and evaluation metrics are tailored to, and take advantage of, the idiosyncracies of medical image analysis and computer-assisted intervention. NiftyNet is built on the TensorFlow framework and supports features such as TensorBoard visualization of 2D and 3D images and computational graphs by default. We present three illustrative medical image analysis applications built using NiftyNet infrastructure: (1) segmentation of multiple abdominal organs from computed tomography; (2) image regression to predict computed tomography attenuation maps from brain magnetic resonance images; and (3) generation of simulated ultrasound images for specified anatomical poses. The NiftyNet infrastructure enables researchers to rapidly develop and distribute deep learning solutions for segmentation, regression, image generation and representation learning applications, or extend the platform to new applications.