VMatcher: State-Space Semi-Dense Local Feature Matching

TL;DR

VMatcher is a hybrid network combining Mamba's efficient state-space model with Transformer attention for semi-dense image feature matching, achieving high accuracy with reduced computational costs.

Contribution

It introduces a novel hybrid architecture that integrates Mamba's linear-complexity model with Transformer attention, enabling efficient and robust semi-dense feature matching.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Demonstrates significant efficiency improvements over traditional Transformer-based methods.

Supports real-time applications with robust matching accuracy.

Abstract

This paper introduces VMatcher, a hybrid Mamba-Transformer network for semi-dense feature matching between image pairs. Learning-based feature matching methods, whether detector-based or detector-free, achieve state-of-the-art performance but depend heavily on the Transformer's attention mechanism, which, while effective, incurs high computational costs due to its quadratic complexity. In contrast, Mamba introduces a Selective State-Space Model (SSM) that achieves comparable or superior performance with linear complexity, offering significant efficiency gains. VMatcher leverages a hybrid approach, integrating Mamba's highly efficient long-sequence processing with the Transformer's attention mechanism. Multiple VMatcher configurations are proposed, including hierarchical architectures, demonstrating their effectiveness in setting new benchmarks efficiently while ensuring robustness and practicality for real-time applications where rapid inference is crucial. Source Code is available at: https://github.com/ayoussf/VMatcher