Unsupervised 3D Keypoint Discovery with Multi-View Geometry

TL;DR

This paper introduces an unsupervised method for discovering 3D human body keypoints from multi-view images, leveraging geometric constraints without requiring manual annotations, resulting in more interpretable and accurate keypoints.

Contribution

It presents a novel algorithm that learns 3D keypoints from multi-view images without supervision, utilizing multi-view geometry and self-estimated masks for meaningful keypoint discovery.

Findings

Outperforms state-of-the-art unsupervised methods on Human3.6M.

Achieves more interpretable 3D keypoints.

Demonstrates robustness without manual annotations.

Abstract

Analyzing and training 3D body posture models depend heavily on the availability of joint labels that are commonly acquired through laborious manual annotation of body joints or via marker-based joint localization using carefully curated markers and capturing systems. However, such annotations are not always available, especially for people performing unusual activities. In this paper, we propose an algorithm that learns to discover 3D keypoints on human bodies from multiple-view images without any supervision or labels other than the constraints multiple-view geometry provides. To ensure that the discovered 3D keypoints are meaningful, they are re-projected to each view to estimate the person's mask that the model itself has initially estimated without supervision. Our approach discovers more interpretable and accurate 3D keypoints compared to other state-of-the-art unsupervised approaches on Human3.6M and MPI-INF-3DHP benchmark datasets.