Weakly Supervised Keypoint Discovery

TL;DR

This paper introduces a weakly supervised method for discovering semantically meaningful keypoints in images with high viewpoint and appearance variation, using image-level labels and viewpoint-equivariance constraints.

Contribution

It proposes a novel approach combining image-level supervision, structural deformation, and viewpoint-equivariance to improve keypoint discovery across diverse images.

Findings

Achieves state-of-the-art performance in weakly supervised keypoint estimation.

Discovered keypoints are directly applicable to downstream tasks.

Effectively handles high viewpoint and appearance variations.

Abstract

In this paper, we propose a method for keypoint discovery from a 2D image using image-level supervision. Recent works on unsupervised keypoint discovery reliably discover keypoints of aligned instances. However, when the target instances have high viewpoint or appearance variation, the discovered keypoints do not match the semantic correspondences over different images. Our work aims to discover keypoints even when the target instances have high viewpoint and appearance variation by using image-level supervision. Motivated by the weakly-supervised learning approach, our method exploits image-level supervision to identify discriminative parts and infer the viewpoint of the target instance. To discover diverse parts, we adopt a conditional image generation approach using a pair of images with structural deformation. Finally, we enforce a viewpoint-based equivariance constraint using the keypoints from the image-level supervision to resolve the spatial correlation problem that consistently appears in the images taken from various viewpoints. Our approach achieves state-of-the-art performance for the task of keypoint estimation on the limited supervision scenarios. Furthermore, the discovered keypoints are directly applicable to downstream tasks without requiring any keypoint labels.