Synth-by-Reg (SbR): Contrastive learning for synthesis-based registration of paired images

TL;DR

This paper introduces Synth-by-Reg, a contrastive learning-based registration method that converts challenging inter-modality registration into an easier intra-modality task, improving accuracy in histology-MRI alignment without requiring perfectly aligned training data.

Contribution

It proposes a novel synthesis-by-registration framework with a weakly supervised loss and contrastive structure preservation, advancing inter-modality image registration techniques.

Findings

13% reduction in landmark error over mutual information methods

11% reduction compared to CycleGAN-based algorithms

Comparable performance to supervised registration CNNs

Abstract

Nonlinear inter-modality registration is often challenging due to the lack of objective functions that are good proxies for alignment. Here we propose a synthesis-by-registration method to convert this problem into an easier intra-modality task. We introduce a registration loss for weakly supervised image translation between domains that does not require perfectly aligned training data. This loss capitalises on a registration U-Net with frozen weights, to drive a synthesis CNN towards the desired translation. We complement this loss with a structure preserving constraint based on contrastive learning, which prevents blurring and content shifts due to overfitting. We apply this method to the registration of histological sections to MRI slices, a key step in 3D histology reconstruction. Results on two different public datasets show improvements over registration based on mutual information (13% reduction in landmark error) and synthesis-based algorithms such as CycleGAN (11% reduction), and are comparable to a registration CNN with label supervision. Code and data are publicly available at \url{https://github.com/acasamitjana/SynthByReg}