SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesis

TL;DR

SyncDiff is a novel diffusion-based method that synthesizes synchronized multi-body human-object interaction motions, addressing the complex correlations among multiple bodies in VR and animation.

Contribution

It introduces a unified diffusion model with frequency-domain decomposition and alignment scores for synchronized multi-body motion synthesis, a significant advancement over prior single-body approaches.

Findings

Outperforms existing methods on four datasets

Effectively captures complex multi-body correlations

Enhances motion synchronization and fidelity

Abstract

Synthesizing realistic human-object interaction motions is a critical problem in VR/AR and human animation. Unlike the commonly studied scenarios involving a single human or hand interacting with one object, we address a more generic multi-body setting with arbitrary numbers of humans, hands, and objects. This complexity introduces significant challenges in synchronizing motions due to the high correlations and mutual influences among bodies. To address these challenges, we introduce SyncDiff, a novel method for multi-body interaction synthesis using a synchronized motion diffusion strategy. SyncDiff employs a single diffusion model to capture the joint distribution of multi-body motions. To enhance motion fidelity, we propose a frequency-domain motion decomposition scheme. Additionally, we introduce a new set of alignment scores to emphasize the synchronization of different body motions. SyncDiff jointly optimizes both data sample likelihood and alignment likelihood through an explicit synchronization strategy. Extensive experiments across four datasets with various multi-body configurations demonstrate the superiority of SyncDiff over existing state-of-the-art motion synthesis methods.