Correspondence Networks with Adaptive Neighbourhood Consensus

TL;DR

This paper introduces ANC-Net, a neural network architecture with adaptive neighbourhood consensus and multi-scale self-similarity modules, designed for dense visual correspondence matching across images with large intra-class variations, trained with sparse annotations.

Contribution

The paper presents a novel end-to-end trainable CNN with non-isotropic 4D convolution and orthogonal loss for robust dense correspondence matching, handling intra-class variations effectively.

Findings

Outperforms state-of-the-art methods on multiple benchmarks.

Robust matching achieved with sparse key-point annotations.

Effective handling of intra-class variations demonstrated.

Abstract

In this paper, we tackle the task of establishing dense visual correspondences between images containing objects of the same category. This is a challenging task due to large intra-class variations and a lack of dense pixel level annotations. We propose a convolutional neural network architecture, called adaptive neighbourhood consensus network (ANC-Net), that can be trained end-to-end with sparse key-point annotations, to handle this challenge. At the core of ANC-Net is our proposed non-isotropic 4D convolution kernel, which forms the building block for the adaptive neighbourhood consensus module for robust matching. We also introduce a simple and efficient multi-scale self-similarity module in ANC-Net to make the learned feature robust to intra-class variations. Furthermore, we propose a novel orthogonal loss that can enforce the one-to-one matching constraint. We thoroughly evaluate the effectiveness of our method on various benchmarks, where it substantially outperforms state-of-the-art methods.