One registration is worth two segmentations

TL;DR

This paper introduces a novel image registration approach using corresponding regions-of-interest (ROI) pairs, leveraging the Segment Anything Model (SAM) for accurate, training-free registration across various medical imaging modalities.

Contribution

It proposes a new ROI-based registration paradigm and integrates SAM for a practical, training-free registration method that outperforms traditional intensity-based and learning-based methods.

Findings

SAMReg accurately segments and matches ROI pairs in medical images.

The method outperforms traditional algorithms in registration accuracy.

SAMReg achieves competitive results with weakly-supervised approaches.

Abstract

The goal of image registration is to establish spatial correspondence between two or more images, traditionally through dense displacement fields (DDFs) or parametric transformations (e.g., rigid, affine, and splines). Rethinking the existing paradigms of achieving alignment via spatial transformations, we uncover an alternative but more intuitive correspondence representation: a set of corresponding regions-of-interest (ROI) pairs, which we demonstrate to have sufficient representational capability as other correspondence representation methods.Further, it is neither necessary nor sufficient for these ROIs to hold specific anatomical or semantic significance. In turn, we formulate image registration as searching for the same set of corresponding ROIs from both moving and fixed images - in other words, two multi-class segmentation tasks on a pair of images. For a general-purpose and practical implementation, we integrate the segment anything model (SAM) into our proposed algorithms, resulting in a SAM-enabled registration (SAMReg) that does not require any training data, gradient-based fine-tuning or engineered prompts. We experimentally show that the proposed SAMReg is capable of segmenting and matching multiple ROI pairs, which establish sufficiently accurate correspondences, in three clinical applications of registering prostate MR, cardiac MR and abdominal CT images. Based on metrics including Dice and target registration errors on anatomical structures, the proposed registration outperforms both intensity-based iterative algorithms and DDF-predicting learning-based networks, even yielding competitive performance with weakly-supervised registration which requires fully-segmented training data.