HumanGenesis: Agent-Based Geometric and Generative Modeling for Synthetic Human Dynamics

TL;DR

HumanGenesis introduces an agent-based framework that combines geometric reconstruction and generative modeling to produce photorealistic, expressive human videos with improved fidelity and scene coherence.

Contribution

The paper presents a novel multi-agent system integrating 3D reconstruction, critique, pose guidance, and video harmonization for synthetic human dynamics.

Findings

Achieves state-of-the-art results in text-guided synthesis and video reenactment.

Significantly improves geometric fidelity and scene integration.

Enhances motion generalization for expressive pose sequences.

Abstract

\textbf{Synthetic human dynamics} aims to generate photorealistic videos of human subjects performing expressive, intention-driven motions. However, current approaches face two core challenges: (1) \emph{geometric inconsistency} and \emph{coarse reconstruction}, due to limited 3D modeling and detail preservation; and (2) \emph{motion generalization limitations} and \emph{scene inharmonization}, stemming from weak generative capabilities. To address these, we present \textbf{HumanGenesis}, a framework that integrates geometric and generative modeling through four collaborative agents: (1) \textbf{Reconstructor} builds 3D-consistent human-scene representations from monocular video using 3D Gaussian Splatting and deformation decomposition. (2) \textbf{Critique Agent} enhances reconstruction fidelity by identifying and refining poor regions via multi-round MLLM-based reflection. (3) \textbf{Pose Guider} enables motion generalization by generating expressive pose sequences using time-aware parametric encoders. (4) \textbf{Video Harmonizer} synthesizes photorealistic, coherent video via a hybrid rendering pipeline with diffusion, refining the Reconstructor through a Back-to-4D feedback loop. HumanGenesis achieves state-of-the-art performance on tasks including text-guided synthesis, video reenactment, and novel-pose generalization, significantly improving expressiveness, geometric fidelity, and scene integration.