Shape-aware Multi-Person Pose Estimation from Multi-View Images

TL;DR

This paper presents a robust multi-view 3D multi-person pose estimation method that combines a coarse-to-fine pipeline with a statistical body model, achieving state-of-the-art results on public datasets.

Contribution

A novel multi-view multi-person pose estimation approach that integrates 2D observations, a confidence-aware voting scheme, and SMPL-based regularization in a joint optimization framework.

Findings

Achieves state-of-the-art performance on public datasets.

Robust to noisy 2D detections and inter-person occlusions.

Generalizes well across different data sources.

Abstract

In this paper we contribute a simple yet effective approach for estimating 3D poses of multiple people from multi-view images. Our proposed coarse-to-fine pipeline first aggregates noisy 2D observations from multiple camera views into 3D space and then associates them into individual instances based on a confidence-aware majority voting technique. The final pose estimates are attained from a novel optimization scheme which links high-confidence multi-view 2D observations and 3D joint candidates. Moreover, a statistical parametric body model such as SMPL is leveraged as a regularizing prior for these 3D joint candidates. Specifically, both 3D poses and SMPL parameters are optimized jointly in an alternating fashion. Here the parametric models help in correcting implausible 3D pose estimates and filling in missing joint detections while updated 3D poses in turn guide obtaining better SMPL estimations. By linking 2D and 3D observations, our method is both accurate and generalizes to different data sources because it better decouples the final 3D pose from the inter-person constellation and is more robust to noisy 2D detections. We systematically evaluate our method on public datasets and achieve state-of-the-art performance. The code and video will be available on the project page: https://ait.ethz.ch/projects/2021/multi-human-pose/.