SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimation

TL;DR

This paper introduces SMAP, a novel single-shot bottom-up method for multi-person 3D pose estimation from a single RGB image that leverages 2.5D body part representations to improve accuracy and contextual reasoning.

Contribution

The paper proposes a new bottom-up approach that regresses 2.5D body part representations and reconstructs 3D poses, outperforming previous top-down methods.

Findings

Achieves state-of-the-art results on CMU Panoptic and MuPoTS-3D datasets.

Effectively models inter-person depth relationships.

Applicable to in-the-wild videos.

Abstract

Recovering multi-person 3D poses with absolute scales from a single RGB image is a challenging problem due to the inherent depth and scale ambiguity from a single view. Addressing this ambiguity requires to aggregate various cues over the entire image, such as body sizes, scene layouts, and inter-person relationships. However, most previous methods adopt a top-down scheme that first performs 2D pose detection and then regresses the 3D pose and scale for each detected person individually, ignoring global contextual cues. In this paper, we propose a novel system that first regresses a set of 2.5D representations of body parts and then reconstructs the 3D absolute poses based on these 2.5D representations with a depth-aware part association algorithm. Such a single-shot bottom-up scheme allows the system to better learn and reason about the inter-person depth relationship, improving both 3D and 2D pose estimation. The experiments demonstrate that the proposed approach achieves the state-of-the-art performance on the CMU Panoptic and MuPoTS-3D datasets and is applicable to in-the-wild videos.