DenseGAP: Graph-Structured Dense Correspondence Learning with Anchor Points

TL;DR

DenseGAP introduces a graph-structured neural network with anchor points to efficiently learn dense correspondences between images, capturing local and global context for improved accuracy and high-resolution feature mapping.

Contribution

The paper proposes a novel graph-based neural network architecture conditioned on anchor points for efficient dense correspondence learning, incorporating a coarse-to-fine cycle consistency framework.

Findings

Outperforms state-of-the-art on large-scale indoor and outdoor datasets.

Efficient high-resolution feature map generation with low memory cost.

Effective global and local context integration improves matching accuracy.

Abstract

Establishing dense correspondence between two images is a fundamental computer vision problem, which is typically tackled by matching local feature descriptors. However, without global awareness, such local features are often insufficient for disambiguating similar regions. And computing the pairwise feature correlation across images is both computation-expensive and memory-intensive. To make the local features aware of the global context and improve their matching accuracy, we introduce DenseGAP, a new solution for efficient Dense correspondence learning with a Graph-structured neural network conditioned on Anchor Points. Specifically, we first propose a graph structure that utilizes anchor points to provide sparse but reliable prior on inter- and intra-image context and propagates them to all image points via directed edges. We also design a graph-structured network to broadcast multi-level contexts via light-weighted message-passing layers and generate high-resolution feature maps at low memory cost. Finally, based on the predicted feature maps, we introduce a coarse-to-fine framework for accurate correspondence prediction using cycle consistency. Our feature descriptors capture both local and global information, thus enabling a continuous feature field for querying arbitrary points at high resolution. Through comprehensive ablative experiments and evaluations on large-scale indoor and outdoor datasets, we demonstrate that our method advances the state-of-the-art of correspondence learning on most benchmarks.