Learning to Match 2D Keypoints Across Preoperative MR and Intraoperative Ultrasound

TL;DR

This paper introduces a texture-invariant 2D keypoints descriptor and a matching-by-synthesis strategy to improve matching between preoperative MR and intraoperative US images, demonstrating superior accuracy.

Contribution

It presents a novel descriptor and synthesis-based matching method tailored for cross-modality image registration in medical imaging.

Findings

Achieves 80.35% matching precision on real cases

Outperforms existing state-of-the-art methods

Robust keypoints descriptors learned through supervised contrastive training

Abstract

We propose in this paper a texture-invariant 2D keypoints descriptor specifically designed for matching preoperative Magnetic Resonance (MR) images with intraoperative Ultrasound (US) images. We introduce a matching-by-synthesis strategy, where intraoperative US images are synthesized from MR images accounting for multiple MR modalities and intraoperative US variability. We build our training set by enforcing keypoints localization over all images then train a patient-specific descriptor network that learns texture-invariant discriminant features in a supervised contrastive manner, leading to robust keypoints descriptors. Our experiments on real cases with ground truth show the effectiveness of the proposed approach, outperforming the state-of-the-art methods and achieving 80.35% matching precision on average.