Image Matching: An Application-oriented Benchmark

TL;DR

This paper introduces a comprehensive benchmark with new metrics and a large dataset to evaluate image matching methods for real-world applications, emphasizing overall performance and practical suitability.

Contribution

It presents a novel application-oriented benchmark with a large-scale dataset and evaluation metrics, filling a gap in assessing image matchers beyond local feature evaluation.

Findings

State-of-the-art methods are evaluated comprehensively.

Application requirements influence matcher performance.

Guidelines for selecting suitable matching approaches.

Abstract

Image matching approaches have been widely used in computer vision applications in which the image-level matching performance of matchers is critical. However, it has not been well investigated by previous works which place more emphases on evaluating local features. To this end, we present a uniform benchmark with novel evaluation metrics and a large-scale dataset for evaluating the overall performance of image matching methods. The proposed metrics are application-oriented as they emphasize application requirements for matchers. The dataset contains two portions for benchmarking video frame matching and unordered image matching separately, where each portion consists of real-world image sequences and each sequence has a specific attribute. Subsequently, we carry out a comprehensive performance evaluation of different state-of-the-art methods and conduct in-depth analyses regarding various aspects such as application requirements, matching types, and data diversity. Moreover, we shed light on how to choose appropriate approaches for different applications based on empirical results and analyses. Conclusions in this benchmark can be used as general guidelines to design practical matching systems and also advocate potential future research directions in this field.