Decoupled Generative Modeling for Human-Object Interaction Synthesis

TL;DR

DecHOI introduces a decoupled approach to synthesize realistic human-object interactions by separating path planning from action generation, improving flexibility and realism in dynamic scene modeling.

Contribution

It proposes a novel decoupled generative framework that separates trajectory planning from action synthesis for more flexible and realistic HOI generation.

Findings

DecHOI outperforms prior methods on FullBodyManipulation and 3D-FUTURE benchmarks.

The approach achieves higher quantitative metrics and qualitative scores.

Perceptual studies favor DecHOI's realistic interaction synthesis.

Abstract

Synthesizing realistic human-object interaction (HOI) is essential for 3D computer vision and robotics, underpinning animation and embodied control. Existing approaches often require manually specified intermediate waypoints and place all optimization objectives on a single network, which increases complexity, reduces flexibility, and leads to errors such as unsynchronized human and object motion or penetration. To address these issues, we propose Decoupled Generative Modeling for Human-Object Interaction Synthesis (DecHOI), which separates path planning and action synthesis. A trajectory generator first produces human and object trajectories without prescribed waypoints, and an action generator conditions on these paths to synthesize detailed motions. To further improve contact realism, we employ adversarial training with a discriminator that focuses on the dynamics of distal joints. The framework also models a moving counterpart and supports responsive, long-sequence planning in dynamic scenes, while preserving plan consistency. Across two benchmarks, FullBodyManipulation and 3D-FUTURE, DecHOI surpasses prior methods on most quantitative metrics and qualitative evaluations, and perceptual studies likewise prefer our results.