CrowdRec: 3D Crowd Reconstruction from Single Color Images

TL;DR

CrowdRec introduces a novel crowd-constrained optimization approach that enhances 3D crowd reconstruction from single color images by refining single-person mesh recovery with crowd features, addressing occlusion and spatial distribution challenges.

Contribution

The paper proposes a crowd-constrained optimization method that improves 3D crowd reconstruction accuracy from monocular images by integrating crowd features into single-person mesh recovery.

Findings

Achieves accurate 3D body poses and shapes in crowded scenes

Effectively refines single-person mesh recovery using crowd constraints

Addresses occlusion and spatial distribution issues in 3D crowd reconstruction

Abstract

This is a technical report for the GigaCrowd challenge. Reconstructing 3D crowds from monocular images is a challenging problem due to mutual occlusions, server depth ambiguity, and complex spatial distribution. Since no large-scale 3D crowd dataset can be used to train a robust model, the current multi-person mesh recovery methods can hardly achieve satisfactory performance in crowded scenes. In this paper, we exploit the crowd features and propose a crowd-constrained optimization to improve the common single-person method on crowd images. To avoid scale variations, we first detect human bounding-boxes and 2D poses from the original images with off-the-shelf detectors. Then, we train a single-person mesh recovery network using existing in-the-wild image datasets. To promote a more reasonable spatial distribution, we further propose a crowd constraint to refine the single-person network parameters. With the optimization, we can obtain accurate body poses and shapes with reasonable absolute positions from a large-scale crowd image using a single-person backbone. The code will be publicly available at~\url{https://github.com/boycehbz/CrowdRec}.