HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perception

TL;DR

HOSIG introduces a hierarchical framework for realistic full-body human-object-scene interaction generation, combining scene-aware grasping, obstacle-avoiding navigation, and detailed motion synthesis to produce plausible, long-duration interactions with minimal manual effort.

Contribution

The paper presents a novel hierarchical approach that integrates scene perception, navigation, and motion diffusion for high-fidelity interaction synthesis, addressing limitations of prior methods.

Findings

Outperforms state-of-the-art on TRUMANS dataset

Supports unlimited motion length via autoregressive generation

Requires minimal manual intervention

Abstract

Generating high-fidelity full-body human interactions with dynamic objects and static scenes remains a critical challenge in computer graphics and animation. Existing methods for human-object interaction often neglect scene context, leading to implausible penetrations, while human-scene interaction approaches struggle to coordinate fine-grained manipulations with long-range navigation. To address these limitations, we propose HOSIG, a novel framework for synthesizing full-body interactions through hierarchical scene perception. Our method decouples the task into three key components: 1) a scene-aware grasp pose generator that ensures collision-free whole-body postures with precise hand-object contact by integrating local geometry constraints, 2) a heuristic navigation algorithm that autonomously plans obstacle-avoiding paths in complex indoor environments via compressed 2D floor maps and dual-component spatial reasoning, and 3) a scene-guided motion diffusion model that generates trajectory-controlled, full-body motions with finger-level accuracy by incorporating spatial anchors and dual-space classifier-free guidance. Extensive experiments on the TRUMANS dataset demonstrate superior performance over state-of-the-art methods. Notably, our framework supports unlimited motion length through autoregressive generation and requires minimal manual intervention. This work bridges the critical gap between scene-aware navigation and dexterous object manipulation, advancing the frontier of embodied interaction synthesis. Codes will be available after publication. Project page: http://yw0208.github.io/hosig