Spatially Covariant Image Registration with Text Prompts

TL;DR

This paper introduces textSCF, a novel image registration method that uses spatially covariant filters and textual anatomical prompts to improve accuracy and efficiency in medical image registration tasks.

Contribution

The work presents a new approach combining visual-language models with spatially covariant filters for deformable image registration, enhancing transferability and structural preservation.

Findings

Outperforms state-of-the-art models in MICCAI Learn2Reg 2021 challenge.

Improves Dice score by 11.3% in abdominal registration.

Reduces network parameters by 89.13% and computational operations by 98.34%.

Abstract

Medical images are often characterized by their structured anatomical representations and spatially inhomogeneous contrasts. Leveraging anatomical priors in neural networks can greatly enhance their utility in resource-constrained clinical settings. Prior research has harnessed such information for image segmentation, yet progress in deformable image registration has been modest. Our work introduces textSCF, a novel method that integrates spatially covariant filters and textual anatomical prompts encoded by visual-language models, to fill this gap. This approach optimizes an implicit function that correlates text embeddings of anatomical regions to filter weights, relaxing the typical translation-invariance constraint of convolutional operations. TextSCF not only boosts computational efficiency but can also retain or improve registration accuracy. By capturing the contextual interplay between anatomical regions, it offers impressive inter-regional transferability and the ability to preserve structural discontinuities during registration. TextSCF's performance has been rigorously tested on inter-subject brain MRI and abdominal CT registration tasks, outperforming existing state-of-the-art models in the MICCAI Learn2Reg 2021 challenge and leading the leaderboard. In abdominal registrations, textSCF's larger model variant improved the Dice score by 11.3% over the second-best model, while its smaller variant maintained similar accuracy but with an 89.13% reduction in network parameters and a 98.34\% decrease in computational operations.