Synth-by-Reg (SbR): Contrastive Learning for Synthesis-Based Registration of Paired Images

• Published in: Simulation and Synthesis in Medical Imaging Vol. 12965; pp. 44 - 54
• Main Authors: Casamitjana, Adrià, Mancini, Matteo, Iglesias, Juan Eugenio
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 01.01.2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Contrastive estimation; Deformable registration; Image synthesis; Inter-modality registration
• ISBN: 3030875911; 9783030875916
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-87592-3_5

Nonlinear inter-modality registration is often challenging due to the lack of objective functions that are good proxies for alignment. Here we propose a synthesis-by-registration method to convert this problem into an easier intra-modality task. We introduce a registration loss for weakly supervised image translation between domains that does not require perfectly aligned training data. This loss capitalises on a registration U-Net with frozen weights, to drive a synthesis CNN towards the desired translation. We complement this loss with a structure preserving constraint based on contrastive learning, which prevents blurring and content shifts due to overfitting. We apply this method to the registration of histological sections to MRI slices, a key step in 3D histology reconstruction. Results on two public datasets show improvements over registration based on mutual information (13% reduction in landmark error) and synthesis-based algorithms such as CycleGAN (11% reduction), and are comparable to registration with label supervision. Code and data are publicly available at https://github.com/acasamitjana/SynthByReg.