Matching neural paths: transfer from recognition to correspondence search

TL;DR

This paper introduces a transfer learning approach using hierarchical neural features to solve low-level correspondence problems, like stereo matching, without requiring labeled data in the target domain.

Contribution

It proposes a novel method to aggregate neural paths for correspondence using a polynomial algorithm, enabling transfer from recognition tasks to correspondence search.

Findings

Achieves competitive stereo correspondence results without target domain labels.

Introduces a polynomial algorithm for neural path aggregation.

Demonstrates effectiveness on low-level matching tasks.

Abstract

Many machine learning tasks require finding per-part correspondences between objects. In this work we focus on low-level correspondences - a highly ambiguous matching problem. We propose to use a hierarchical semantic representation of the objects, coming from a convolutional neural network, to solve this ambiguity. Training it for low-level correspondence prediction directly might not be an option in some domains where the ground-truth correspondences are hard to obtain. We show how transfer from recognition can be used to avoid such training. Our idea is to mark parts as "matching" if their features are close to each other at all the levels of convolutional feature hierarchy (neural paths). Although the overall number of such paths is exponential in the number of layers, we propose a polynomial algorithm for aggregating all of them in a single backward pass. The empirical validation is done on the task of stereo correspondence and demonstrates that we achieve competitive results among the methods which do not use labeled target domain data.