Semantic Correspondence: A Hierarchical Approach

TL;DR

This paper introduces a hierarchical method for establishing semantic correspondence across images with large object deformations, improving accuracy by semantic localization and multi-level feature analysis.

Contribution

It presents a novel hierarchical approach that localizes search space semantically and leverages multi-level features to handle large deformations in semantic correspondence tasks.

Findings

Outperforms state-of-the-art methods in semantic correspondence accuracy

Robustly handles large object deformations

Utilizes hypercolumn features for improved localization

Abstract

Establishing semantic correspondence across images when the objects in the images have undergone complex deformations remains a challenging task in the field of computer vision. In this paper, we propose a hierarchical method to tackle this problem by first semantically targeting the foreground objects to localize the search space and then looking deeply into multiple levels of the feature representation to search for point-level correspondence. In contrast to existing approaches, which typically penalize large discrepancies, our approach allows for significant displacements, with the aim to accommodate large deformations of the objects in scene. Localizing the search space by semantically matching object-level correspondence, our method robustly handles large deformations of objects. Representing the target region by concatenated hypercolumn features which take into account the hierarchical levels of the surrounding context, helps to clear the ambiguity to further improve the accuracy. By conducting multiple experiments across scenes with non-rigid objects, we validate the proposed approach, and show that it outperforms the state of the art methods for semantic correspondence establishment.