$\mathbb{X}$Resolution Correspondence Networks

TL;DR

This paper introduces a simplified dense correspondence network that removes computationally intensive modules, enabling faster training and higher resolution testing, ultimately surpassing state-of-the-art accuracy on benchmarks.

Contribution

The authors demonstrate that removing the 4D correlation tensor from correspondence networks has minimal impact on accuracy, significantly speeding up training and enabling high-resolution testing.

Findings

Removing 4D correlation tensor speeds up training.

Optimal resolution improves matching accuracy.

Proposed method outperforms state-of-the-art benchmarks.

Abstract

In this paper, we aim at establishing accurate dense correspondences between a pair of images with overlapping field of view under challenging illumination variation, viewpoint changes, and style differences. Through an extensive ablation study of the state-of-the-art correspondence networks, we surprisingly discovered that the widely adopted 4D correlation tensor and its related learning and processing modules could be de-parameterised and removed from training with merely a minor impact over the final matching accuracy. Disabling these computational expensive modules dramatically speeds up the training procedure and allows to use 4 times bigger batch size, which in turn compensates for the accuracy drop. Together with a multi-GPU inference stage, our method facilitates the systematic investigation of the relationship between matching accuracy and up-sampling resolution of the native testing images from 1280 to 4K. This leads to discovery of the existence of an optimal resolution $\mathbb{X}$ that produces accurate matching performance surpassing the state-of-the-art methods particularly over the lower error band on public benchmarks for the proposed network.