Semantic Matching by Weakly Supervised 2D Point Set Registration

TL;DR

This paper introduces a weakly supervised CNN-based method for semantic matching that aligns object instances by regressing geometric transformation parameters using a novel cyclic consistency loss, achieving state-of-the-art results.

Contribution

It proposes a novel weakly supervised approach using a cyclic consistency loss for 2D point set registration without explicit correspondence labels.

Findings

Achieves state-of-the-art results on PF-PASCAL dataset.

Benefits from additional training data generated with category information.

Effective in aligning semantic keypoints across object instances.

Abstract

In this paper we address the problem of establishing correspondences between different instances of the same object. The problem is posed as finding the geometric transformation that aligns a given image pair. We use a convolutional neural network (CNN) to directly regress the parameters of the transformation model. The alignment problem is defined in the setting where an unordered set of semantic key-points per image are available, but, without the correspondence information. To this end we propose a novel loss function based on cyclic consistency that solves this 2D point set registration problem by inferring the optimal geometric transformation model parameters. We train and test our approach on a standard benchmark dataset Proposal-Flow (PF-PASCAL)\cite{proposal_flow}. The proposed approach achieves state-of-the-art results demonstrating the effectiveness of the method. In addition, we show our approach further benefits from additional training samples in PF-PASCAL generated by using category level information.