SuperPoint features in endoscopy

TL;DR

This paper evaluates the performance of SuperPoint, a self-supervised local feature extractor, on endoscopic images, demonstrating its superiority over traditional features like SIFT for medical image matching and 3D reconstruction.

Contribution

It adapts the SuperPoint model for endoscopic images, introduces a new evaluation framework, and shows improved matching quality and robustness against artifacts.

Findings

SuperPoint outperforms traditional local features in endoscopy.

The adapted model reduces errors caused by specularity regions.

Enhanced matching improves 3D reconstruction quality.

Abstract

There is often a significant gap between research results and applicability in routine medical practice. This work studies the performance of well-known local features on a medical dataset captured during routine colonoscopy procedures. Local feature extraction and matching is a key step for many computer vision applications, specially regarding 3D modelling. In the medical domain, handcrafted local features such as SIFT, with public pipelines such as COLMAP, are still a predominant tool for this kind of tasks. We explore the potential of the well known self-supervised approach SuperPoint, present an adapted variation for the endoscopic domain and propose a challenging evaluation framework. SuperPoint based models achieve significantly higher matching quality than commonly used local features in this domain. Our adapted model avoids features within specularity regions, a frequent and problematic artifact in endoscopic images, with consequent benefits for matching and reconstruction results.