Deep Matching Prior: Test-Time Optimization for Dense Correspondence

TL;DR

Deep Matching Prior (DMP) introduces a test-time optimization approach that adapts untrained matching networks to specific image pairs, achieving high accuracy without extensive training data.

Contribution

The paper proposes a novel test-time optimization method for dense correspondence that captures pair-specific priors using untrained networks, outperforming existing learning-based methods.

Findings

DMP achieves state-of-the-art results on multiple benchmarks.

DMP outperforms recent learning-based methods without large training data.

Pre-trained DMP attains competitive performance across tasks.

Abstract

Conventional techniques to establish dense correspondences across visually or semantically similar images focused on designing a task-specific matching prior, which is difficult to model. To overcome this, recent learning-based methods have attempted to learn a good matching prior within a model itself on large training data. The performance improvement was apparent, but the need for sufficient training data and intensive learning hinders their applicability. Moreover, using the fixed model at test time does not account for the fact that a pair of images may require their own prior, thus providing limited performance and poor generalization to unseen images. In this paper, we show that an image pair-specific prior can be captured by solely optimizing the untrained matching networks on an input pair of images. Tailored for such test-time optimization for dense correspondence, we present a residual matching network and a confidence-aware contrastive loss to guarantee a meaningful convergence. Experiments demonstrate that our framework, dubbed Deep Matching Prior (DMP), is competitive, or even outperforms, against the latest learning-based methods on several benchmarks for geometric matching and semantic matching, even though it requires neither large training data nor intensive learning. With the networks pre-trained, DMP attains state-of-the-art performance on all benchmarks.