Unsupervised Feature Learning for Dense Correspondences across Scenes

TL;DR

This paper introduces a fast, accurate unsupervised feature learning approach for dense pixel correspondence across scenes, outperforming traditional hand-crafted features and existing methods in accuracy and efficiency.

Contribution

It presents a novel unsupervised learning framework for pixel features integrated into a multi-layer matching model, improving dense correspondence estimation across scenes.

Findings

Learned features outperform SIFT in accuracy.

The method is faster than state-of-the-art techniques.

Analysis of dictionary learning impacts on matching performance.

Abstract

We propose a fast, accurate matching method for estimating dense pixel correspondences across scenes. It is a challenging problem to estimate dense pixel correspondences between images depicting different scenes or instances of the same object category. While most such matching methods rely on hand-crafted features such as SIFT, we learn features from a large amount of unlabeled image patches using unsupervised learning. Pixel-layer features are obtained by encoding over the dictionary, followed by spatial pooling to obtain patch-layer features. The learned features are then seamlessly embedded into a multi-layer match- ing framework. We experimentally demonstrate that the learned features, together with our matching model, outperforms state-of-the-art methods such as the SIFT flow, coherency sensitive hashing and the recent deformable spatial pyramid matching methods both in terms of accuracy and computation efficiency. Furthermore, we evaluate the performance of a few different dictionary learning and feature encoding methods in the proposed pixel correspondences estimation framework, and analyse the impact of dictionary learning and feature encoding with respect to the final matching performance.