Body Size and Depth Disambiguation in Multi-Person Reconstruction from Single Images

TL;DR

This paper introduces a novel optimization method for multi-person 3D body reconstruction from a single image that effectively handles size variability and depth ambiguity, improving accuracy over existing methods.

Contribution

It proposes a new optimization scheme that learns body scale and camera pose while enforcing ground contact, enhancing multi-person 3D reconstruction accuracy.

Findings

Improves body size and depth estimation in multi-person scenes.

Achieves state-of-the-art results on MuPoTS-3D and 3DPW datasets.

Effectively handles scenes with diverse human heights.

Abstract

We address the problem of multi-person 3D body pose and shape estimation from a single image. While this problem can be addressed by applying single-person approaches multiple times for the same scene, recent works have shown the advantages of building upon deep architectures that simultaneously reason about all people in the scene in a holistic manner by enforcing, e.g., depth order constraints or minimizing interpenetration among reconstructed bodies. However, existing approaches are still unable to capture the size variability of people caused by the inherent body scale and depth ambiguity. In this work, we tackle this challenge by devising a novel optimization scheme that learns the appropriate body scale and relative camera pose, by enforcing the feet of all people to remain on the ground floor. A thorough evaluation on MuPoTS-3D and 3DPW datasets demonstrates that our approach is able to robustly estimate the body translation and shape of multiple people while retrieving their spatial arrangement, consistently improving current state-of-the-art, especially in scenes with people of very different heights