Can 3D Pose be Learned from 2D Projections Alone?

TL;DR

This paper introduces a weakly supervised method for 3D human pose estimation from 2D landmarks using an adversarial framework and a novel random projection layer, eliminating the need for explicit 3D priors or 2D-3D correspondences.

Contribution

It presents a novel adversarial approach with a random projection layer that learns 3D pose from 2D projections without explicit 3D priors or correspondence supervision.

Findings

Outperforms previous supervised and weakly supervised methods on Human3.6M.

Uses a novel Random Projection layer to improve 3D pose estimation.

Demonstrates effectiveness of weak supervision in 3D pose learning.

Abstract

3D pose estimation from a single image is a challenging task in computer vision. We present a weakly supervised approach to estimate 3D pose points, given only 2D pose landmarks. Our method does not require correspondences between 2D and 3D points to build explicit 3D priors. We utilize an adversarial framework to impose a prior on the 3D structure, learned solely from their random 2D projections. Given a set of 2D pose landmarks, the generator network hypothesizes their depths to obtain a 3D skeleton. We propose a novel Random Projection layer, which randomly projects the generated 3D skeleton and sends the resulting 2D pose to the discriminator. The discriminator improves by discriminating between the generated poses and pose samples from a real distribution of 2D poses. Training does not require correspondence between the 2D inputs to either the generator or the discriminator. We apply our approach to the task of 3D human pose estimation. Results on Human3.6M dataset demonstrates that our approach outperforms many previous supervised and weakly supervised approaches.