CoopDiff: Anticipating 3D Human-object Interactions via Contact-consistent Decoupled Diffusion

TL;DR

CoopDiff introduces a contact-consistent decoupled diffusion framework for predicting future 3D human-object interactions by modeling human and object motions separately yet coherently through shared contact points.

Contribution

The paper presents a novel decoupled diffusion approach with contact-based bridging and a human-driven interaction module for improved 3D HOI anticipation.

Findings

Outperforms state-of-the-art methods on BEHAVE and HOI datasets.

Effectively models human and object motions separately with contact consistency.

Enhances prediction reliability through human-driven interaction guidance.

Abstract

3D human-object interaction (HOI) anticipation aims to predict the future motion of humans and their manipulated objects, conditioned on the historical context. Generally, the articulated humans and rigid objects exhibit different motion patterns, due to their distinct intrinsic physical properties. However, this distinction is ignored by most of the existing works, which intend to capture the dynamics of both humans and objects within a single prediction model. In this work, we propose a novel contact-consistent decoupled diffusion framework CoopDiff, which employs two distinct branches to decouple human and object motion modeling, with the human-object contact points as shared anchors to bridge the motion generation across branches. The human dynamics branch is aimed to predict highly structured human motion, while the object dynamics branch focuses on the object motion with rigid translations and rotations. These two branches are bridged by a series of shared contact points with consistency constraint for coherent human-object motion prediction. To further enhance human-object consistency and prediction reliability, we propose a human-driven interaction module to guide object motion modeling. Extensive experiments on the BEHAVE and Human-object Interaction datasets demonstrate that our CoopDiff outperforms state-of-the-art methods.