Learning from Abstract Images: on the Importance of Occlusion in a Minimalist Encoding of Human Poses

TL;DR

This paper introduces a novel 3D pose representation using opaque limbs that encode occlusion and joint information, enabling better cross-dataset generalization and viewpoint independence in pose estimation.

Contribution

The authors propose a new occlusion-aware limb representation for 3D pose lifting, allowing training on synthetic images and improving cross-dataset performance.

Findings

Significant improvement in cross-dataset benchmarks.

Enhanced robustness to occlusion in pose estimation.

Ability to train on synthetic data with arbitrary viewpoints.

Abstract

Existing 2D-to-3D pose lifting networks suffer from poor performance in cross-dataset benchmarks. Although the use of 2D keypoints joined by "stick-figure" limbs has shown promise as an intermediate step, stick-figures do not account for occlusion information that is often inherent in an image. In this paper, we propose a novel representation using opaque 3D limbs that preserves occlusion information while implicitly encoding joint locations. Crucially, when training on data with accurate three-dimensional keypoints and without part-maps, this representation allows training on abstract synthetic images, with occlusion, from as many synthetic viewpoints as desired. The result is a pose defined by limb angles rather than joint positions $\unicode{x2013}$ because poses are, in the real world, independent of cameras $\unicode{x2013}$ allowing us to predict poses that are completely independent of camera viewpoint. The result provides not only an improvement in same-dataset benchmarks, but a "quantum leap" in cross-dataset benchmarks.