Global Multi-modal 2D/3D Registration via Local Descriptors Learning

TL;DR

This paper introduces a novel, fully automatic multi-modal 2D/3D registration method using learned dense keypoint descriptors, overcoming initialization issues and applicable to ultrasound-guided interventions.

Contribution

It presents a new approach that learns dense descriptors for ultrasound-to-MRI registration, eliminating the need for initialization and improving robustness in challenging clinical scenarios.

Findings

Method is fast and fully automatic.

Effective on clinical ultrasound and MRI data.

Overcomes challenges of multi-modality and low training data.

Abstract

Multi-modal registration is a required step for many image-guided procedures, especially ultrasound-guided interventions that require anatomical context. While a number of such registration algorithms are already available, they all require a good initialization to succeed due to the challenging appearance of ultrasound images and the arbitrary coordinate system they are acquired in. In this paper, we present a novel approach to solve the problem of registration of an ultrasound sweep to a pre-operative image. We learn dense keypoint descriptors from which we then estimate the registration. We show that our method overcomes the challenges inherent to registration tasks with freehand ultrasound sweeps, namely, the multi-modality and multidimensionality of the data in addition to lack of precise ground truth and low amounts of training examples. We derive a registration method that is fast, generic, fully automatic, does not require any initialization and can naturally generate visualizations aiding interpretability and explainability. Our approach is evaluated on a clinical dataset of paired MR volumes and ultrasound sequences.