Semi-Supervised Learning of Semantic Correspondence with Pseudo-Labels

TL;DR

This paper introduces SemiMatch, a semi-supervised learning framework for semantic correspondence that leverages pseudo-labels and confidence measures to improve robustness and achieve state-of-the-art results.

Contribution

The paper proposes a novel semi-supervised approach using pseudo-labels and confidence measures for semantic correspondence, enhancing performance with limited ground-truth data.

Findings

Achieves state-of-the-art results on PF-Willow benchmark.

Significantly improves robustness of semantic correspondence models.

Effective use of pseudo-labels reduces reliance on manual annotations.

Abstract

Establishing dense correspondences across semantically similar images remains a challenging task due to the significant intra-class variations and background clutters. Traditionally, a supervised learning was used for training the models, which required tremendous manually-labeled data, while some methods suggested a self-supervised or weakly-supervised learning to mitigate the reliance on the labeled data, but with limited performance. In this paper, we present a simple, but effective solution for semantic correspondence that learns the networks in a semi-supervised manner by supplementing few ground-truth correspondences via utilization of a large amount of confident correspondences as pseudo-labels, called SemiMatch. Specifically, our framework generates the pseudo-labels using the model's prediction itself between source and weakly-augmented target, and uses pseudo-labels to learn the model again between source and strongly-augmented target, which improves the robustness of the model. We also present a novel confidence measure for pseudo-labels and data augmentation tailored for semantic correspondence. In experiments, SemiMatch achieves state-of-the-art performance on various benchmarks, especially on PF-Willow by a large margin.