A Keypoint Detection and Description Network Based on the Vessel Structure for Multi-Modal Retinal Image Registration

TL;DR

This paper introduces a neural network-based method for aligning multi-modal retinal images by detecting and describing vessel keypoints, improving registration accuracy across different imaging modalities.

Contribution

A novel convolutional neural network jointly trained for keypoint detection and description tailored for multi-modal retinal image registration.

Findings

Achieves superior registration performance on multiple datasets.

Outperforms existing methods in multi-modal retinal image alignment.

Effective in handling diverse imaging modalities.

Abstract

Ophthalmological imaging utilizes different imaging systems, such as color fundus, infrared, fluorescein angiography, optical coherence tomography (OCT) or OCT angiography. Multiple images with different modalities or acquisition times are often analyzed for the diagnosis of retinal diseases. Automatically aligning the vessel structures in the images by means of multi-modal registration can support the ophthalmologists in their work. Our method uses a convolutional neural network to extract features of the vessel structure in multi-modal retinal images. We jointly train a keypoint detection and description network on small patches using a classification and a cross-modal descriptor loss function and apply the network to the full image size in the test phase. Our method demonstrates the best registration performance on our and a public multi-modal dataset in comparison to competing methods.