General Vision Encoder Features as Guidance in Medical Image Registration

TL;DR

This paper evaluates the effectiveness of general vision encoder features, including one fine-tuned on medical data, as guidance in improving the accuracy of cardiac MRI registration within a B-spline FFD framework.

Contribution

It provides an in-depth comparison of different general vision encoders for medical image registration and demonstrates their potential to enhance registration quality.

Findings

Features from vision encoders improve registration accuracy

Fine-tuned medical encoder outperforms natural image encoders

Guided metrics lead to better registration results

Abstract

General vision encoders like DINOv2 and SAM have recently transformed computer vision. Even though they are trained on natural images, such encoder models have excelled in medical imaging, e.g., in classification, segmentation, and registration. However, no in-depth comparison of different state-of-the-art general vision encoders for medical registration is available. In this work, we investigate how well general vision encoder features can be used in the dissimilarity metrics for medical image registration. We explore two encoders that were trained on natural images as well as one that was fine-tuned on medical data. We apply the features within the well-established B-spline FFD registration framework. In extensive experiments on cardiac cine MRI data, we find that using features as additional guidance for conventional metrics improves the registration quality. The code is available at github.com/compai-lab/2024-miccai-koegl.