UnrealPose: Leveraging Game Engine Kinematics for Large-Scale Synthetic Human Pose Data

TL;DR

UnrealPose-Gen is a pipeline using Unreal Engine 5 to generate large-scale, high-quality synthetic human pose data with detailed annotations, aiding research in pose estimation and related tasks.

Contribution

We introduce UnrealPose-Gen, a novel Unreal Engine 5 pipeline for creating large-scale, annotated synthetic human pose datasets with diverse scenes and camera viewpoints.

Findings

High-quality synthetic data improves pose estimation tasks

UnrealPose-1M dataset covers diverse actions and viewpoints

Pipeline enables customizable generation of human pose data

Abstract

Diverse, accurately labeled 3D human pose data is expensive and studio-bound, while in-the-wild datasets lack known ground truth. We introduce UnrealPose-Gen, an Unreal Engine 5 pipeline built on Movie Render Queue for high-quality offline rendering. Our generated frames include: (i) 3D joints in world and camera coordinates, (ii) 2D projections and COCO-style keypoints with occlusion and joint-visibility flags, (iii) person bounding boxes, and (iv) camera intrinsics and extrinsics. We use UnrealPose-Gen to present UnrealPose-1M, an approximately one million frame corpus comprising eight sequences: five scripted "coherent" sequences spanning five scenes, approximately 40 actions, and five subjects; and three randomized sequences across three scenes, approximately 100 actions, and five subjects, all captured from diverse camera trajectories for broad viewpoint coverage. As a fidelity check, we report real-to-synthetic results on four tasks: image-to-3D pose, 2D keypoint detection, 2D-to-3D lifting, and person detection/segmentation. Though time and resources constrain us from an unlimited dataset, we release the UnrealPose-1M dataset, as well as the UnrealPose-Gen pipeline to support third-party generation of human pose data.