ECO-TR: Efficient Correspondences Finding Via Coarse-to-Fine Refinement

TL;DR

ECO-TR introduces a fast, coarse-to-fine transformer-based method for image correspondence that enhances efficiency without sacrificing accuracy, suitable for real-world applications.

Contribution

The paper presents ECO-TR, a novel transformer architecture with stage-wise refinement and adaptive strategies, improving efficiency in image correspondence tasks.

Findings

Outperforms existing methods in speed and accuracy

Effective in both sparse and dense matching scenarios

Utilizes a single feed-forward pass for predictions

Abstract

Modeling sparse and dense image matching within a unified functional correspondence model has recently attracted increasing research interest. However, existing efforts mainly focus on improving matching accuracy while ignoring its efficiency, which is crucial for realworld applications. In this paper, we propose an efficient structure named Efficient Correspondence Transformer (ECO-TR) by finding correspondences in a coarse-to-fine manner, which significantly improves the efficiency of functional correspondence model. To achieve this, multiple transformer blocks are stage-wisely connected to gradually refine the predicted coordinates upon a shared multi-scale feature extraction network. Given a pair of images and for arbitrary query coordinates, all the correspondences are predicted within a single feed-forward pass. We further propose an adaptive query-clustering strategy and an uncertainty-based outlier detection module to cooperate with the proposed framework for faster and better predictions. Experiments on various sparse and dense matching tasks demonstrate the superiority of our method in both efficiency and effectiveness against existing state-of-the-arts.