DFM: A Performance Baseline for Deep Feature Matching

TL;DR

This paper introduces DFM, a deep feature matching baseline using pre-trained VGG features and a hierarchical alignment inspired by mental rotation, achieving state-of-the-art accuracy without additional training.

Contribution

The method leverages pre-trained deep features and a novel hierarchical alignment inspired by psychology, providing a strong performance baseline for image matching.

Findings

Achieves 0.57 MMA at 1 pixel threshold

Achieves 0.80 MMA at 2 pixels threshold

Outperforms state-of-the-art on Hpatches dataset

Abstract

A novel image matching method is proposed that utilizes learned features extracted by an off-the-shelf deep neural network to obtain a promising performance. The proposed method uses pre-trained VGG architecture as a feature extractor and does not require any additional training specific to improve matching. Inspired by well-established concepts in the psychology area, such as the Mental Rotation paradigm, an initial warping is performed as a result of a preliminary geometric transformation estimate. These estimates are simply based on dense matching of nearest neighbors at the terminal layer of VGG network outputs of the images to be matched. After this initial alignment, the same approach is repeated again between reference and aligned images in a hierarchical manner to reach a good localization and matching performance. Our algorithm achieves 0.57 and 0.80 overall scores in terms of Mean Matching Accuracy (MMA) for 1 pixel and 2 pixels thresholds respectively on Hpatches dataset, which indicates a better performance than the state-of-the-art.