Graph Self-Supervised Learning for Endoscopic Image Matching

TL;DR

This paper introduces a self-supervised learning method combining CNNs and Graph Neural Networks to improve feature matching in endoscopic images, addressing challenges of texture lack and variability.

Contribution

The paper presents a novel self-supervised framework that integrates CNNs and attention-based GNNs for robust endoscopic image matching without labeled data.

Findings

Achieves a matching score of 99.3%

Outperforms existing handcrafted and deep learning methods

Demonstrates high precision in feature matching

Abstract

Accurate feature matching and correspondence in endoscopic images play a crucial role in various clinical applications, including patient follow-up and rapid anomaly localization through panoramic image generation. However, developing robust and accurate feature matching techniques faces challenges due to the lack of discriminative texture and significant variability between patients. To address these limitations, we propose a novel self-supervised approach that combines Convolutional Neural Networks for capturing local visual appearance and attention-based Graph Neural Networks for modeling spatial relationships between key-points. Our approach is trained in a fully self-supervised scheme without the need for labeled data. Our approach outperforms state-of-the-art handcrafted and deep learning-based methods, demonstrating exceptional performance in terms of precision rate (1) and matching score (99.3%). We also provide code and materials related to this work, which can be accessed at https://github.com/abenhamadou/graph-self-supervised-learning-for-endoscopic-image-matching.