Atlas-ISTN: Joint segmentation, registration and atlas construction with image-and-spatial transformer networks

• Published in: Medical image analysis Vol. 78; p. 102383
• Main Authors: Sinclair, Matthew, Schuh, Andreas, Hahn, Karl, Petersen, Kersten, Bai, Ying, Batten, James, Schaap, Michiel, Glocker, Ben
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01.05.2022; Elsevier BV
• Subjects: Brain - diagnostic imaging; Computed tomography; Deep learning; Endoscopy; Heart; Humans; Image processing; Image Processing, Computer-Assisted - methods; Image registration; Image segmentation; Image segmentation and registration; Imaging, Three-Dimensional - methods; Magnetic resonance imaging; Magnetic Resonance Imaging - methods; Medical imaging; Multi-label atlas construction; Network topologies; Neuroimaging; Noise; Noise prediction; Noise sensitivity; Pixels; Predictions; Registration; Spatial discrimination learning; Testing time; Transformers; Image segmentation and registration; Multi-label atlas construction
• ISSN: 1361-8415; 1361-8423; 1361-8423
• DOI: 10.1016/j.media.2022.102383

•A deep learning framework for atlas construction, registration and segmentation.•Generates a population-derived atlas within a deep learning framework.•A robust segmentation system which preserves topology of test time predictions.•Inter-subject correspondence is provided by mapping subjects to atlas space. [Display omitted] Deep learning models for semantic segmentation are able to learn powerful representations for pixel-wise predictions, but are sensitive to noise at test time and may lead to implausible topologies. Image registration models on the other hand are able to warp known topologies to target images as a means of segmentation, but typically require large amounts of training data, and have not widely been benchmarked against pixel-wise segmentation models. We propose the Atlas Image-and-Spatial Transformer Network (Atlas-ISTN), a framework that jointly learns segmentation and registration on 2D and 3D image data, and constructs a population-derived atlas in the process. Atlas-ISTN learns to segment multiple structures of interest and to register the constructed atlas labelmap to an intermediate pixel-wise segmentation. Additionally, Atlas-ISTN allows for test time refinement of the model’s parameters to optimize the alignment of the atlas labelmap to an intermediate pixel-wise segmentation. This process both mitigates for noise in the target image that can result in spurious pixel-wise predictions, as well as improves upon the one-pass prediction of the model. Benefits of the Atlas-ISTN framework are demonstrated qualitatively and quantitatively on 2D synthetic data and 3D cardiac computed tomography and brain magnetic resonance image data, out-performing both segmentation and registration baseline models. Atlas-ISTN also provides inter-subject correspondence of the structures of interest.