RF-Net: An End-to-End Image Matching Network based on Receptive Field

TL;DR

RF-Net is an end-to-end trainable image matching network that leverages receptive fields and a novel neighbor mask loss to improve keypoint detection and descriptor stability, outperforming previous state-of-the-art methods.

Contribution

The paper introduces RF-Net, a new receptive field-based architecture with a neighbor mask loss, enhancing keypoint detection and training stability in image matching.

Findings

RF-Net outperforms existing methods on multiple benchmarks.

Receptive feature maps improve keypoint detection.

Neighbor mask loss stabilizes descriptor training.

Abstract

This paper proposes a new end-to-end trainable matching network based on receptive field, RF-Net, to compute sparse correspondence between images. Building end-to-end trainable matching framework is desirable and challenging. The very recent approach, LF-Net, successfully embeds the entire feature extraction pipeline into a jointly trainable pipeline, and produces the state-of-the-art matching results. This paper introduces two modifications to the structure of LF-Net. First, we propose to construct receptive feature maps, which lead to more effective keypoint detection. Second, we introduce a general loss function term, neighbor mask, to facilitate training patch selection. This results in improved stability in descriptor training. We trained RF-Net on the open dataset HPatches, and compared it with other methods on multiple benchmark datasets. Experiments show that RF-Net outperforms existing state-of-the-art methods.