Gen4D: Synthesizing Humans and Scenes in the Wild

TL;DR

Gen4D is an automated pipeline that creates diverse, photorealistic 4D human animations for sports, enabling scalable synthetic datasets without manual modeling, to improve in-the-wild human-centric vision tasks.

Contribution

We introduce Gen4D, a fully automated, diverse, and photorealistic 4D human animation generation pipeline, and SportPAL, a large-scale synthetic sports dataset, advancing synthetic data for human-centric vision tasks.

Findings

Gen4D produces highly varied, lifelike human sequences.

SportPAL offers a large-scale, diverse sports dataset.

The approach reduces reliance on manual 3D modeling.

Abstract

Lack of input data for in-the-wild activities often results in low performance across various computer vision tasks. This challenge is particularly pronounced in uncommon human-centric domains like sports, where real-world data collection is complex and impractical. While synthetic datasets offer a promising alternative, existing approaches typically suffer from limited diversity in human appearance, motion, and scene composition due to their reliance on rigid asset libraries and hand-crafted rendering pipelines. To address this, we introduce Gen4D, a fully automated pipeline for generating diverse and photorealistic 4D human animations. Gen4D integrates expert-driven motion encoding, prompt-guided avatar generation using diffusion-based Gaussian splatting, and human-aware background synthesis to produce highly varied and lifelike human sequences. Based on Gen4D, we present SportPAL, a large-scale synthetic dataset spanning three sports: baseball, icehockey, and soccer. Together, Gen4D and SportPAL provide a scalable foundation for constructing synthetic datasets tailored to in-the-wild human-centric vision tasks, with no need for manual 3D modeling or scene design.