Sparse Over-complete Patch Matching

TL;DR

This paper introduces a novel patch matching method that leverages sparse over-complete representations of image patches, combined with neural networks, to improve matching accuracy over existing techniques.

Contribution

The paper proposes a new paradigm using sparse coding of image patches as input to neural networks, outperforming state-of-the-art methods on benchmark datasets.

Findings

Achieved new benchmark results on UBC patch datasets.

Outperformed all existing patch matching techniques on evaluated datasets.

Demonstrated the effectiveness of sparse over-complete representations for patch matching.

Abstract

Image patch matching, which is the process of identifying corresponding patches across images, has been used as a subroutine for many computer vision and image processing tasks. State -of-the-art patch matching techniques take image patches as input to a convolutional neural network to extract the patch features and evaluate their similarity. Our aim in this paper is to improve on the state of the art patch matching techniques by observing the fact that a sparse-overcomplete representation of an image posses statistical properties of natural visual scenes which can be exploited for patch matching. We propose a new paradigm which encodes image patch details by encoding the patch and subsequently using this sparse representation as input to a neural network to compare the patches. As sparse coding is based on a generative model of natural image patches, it can represent the patch in terms of the fundamental visual components from which it has been composed of, leading to similar sparse codes for patches which are built from similar components. Once the sparse coded features are extracted, we employ a fully-connected neural network, which captures the non-linear relationships between features, for comparison. We have evaluated our approach using the Liberty and Notredame subsets of the popular UBC patch dataset and set a new benchmark outperforming all state-of-the-art patch matching techniques for these datasets.