CrowdGaussian: Reconstructing High-Fidelity 3D Gaussians for Human Crowd from a Single Image

TL;DR

CrowdGaussian introduces a novel framework for reconstructing high-fidelity 3D models of human crowds from a single image, effectively handling occlusions and diverse appearances through self-supervised learning and diffusion-based refinement.

Contribution

The paper presents a unified approach for single-image multi-person 3D reconstruction using 3D Gaussian Splatting, with new self-supervised adaptation and Self-Calibrated Learning strategies.

Findings

Produces photorealistic, geometrically coherent multi-person 3D reconstructions

Effectively handles occlusions and diverse appearances in crowded scenes

Outperforms existing methods in quality and realism

Abstract

Single-view 3D human reconstruction has garnered significant attention in recent years. Despite numerous advancements, prior research has concentrated on reconstructing 3D models from clear, close-up images of individual subjects, often yielding subpar results in the more prevalent multi-person scenarios. Reconstructing 3D human crowd models is a highly intricate task, laden with challenges such as: 1) extensive occlusions, 2) low clarity, and 3) numerous and various appearances. To address this task, we propose CrowdGaussian, a unified framework that directly reconstructs multi-person 3D Gaussian Splatting (3DGS) representations from single-image inputs. To handle occlusions, we devise a self-supervised adaptation pipeline that enables the pretrained large human model to reconstruct complete 3D humans with plausible geometry and appearance from heavily occluded inputs. Furthermore, we introduce Self-Calibrated Learning (SCL). This training strategy enables single-step diffusion models to adaptively refine coarse renderings to optimal quality by blending identity-preserving samples with clean/corrupted image pairs. The outputs can be distilled back to enhance the quality of multi-person 3DGS representations. Extensive experiments demonstrate that CrowdGaussian generates photorealistic, geometrically coherent reconstructions of multi-person scenes.