Contact Matrix: Enhancing Dance Motion Synthesis with Precise Interaction Modeling

TL;DR

This paper introduces a two-stage framework for realistic duet dance motion synthesis, combining a motion VQ-VAE with a contact-aware diffusion model to improve interaction fidelity and synchronization.

Contribution

The novel approach integrates a specialized motion VQ-VAE with a contact matrix-guided diffusion model for enhanced reactive dance motion generation.

Findings

Outperforms Duolando with lower FID_k and FID_cd scores.

Achieves higher BED, indicating better interaction fidelity.

Provides more precise and constrained interaction dynamics.

Abstract

Generating realistic reactive motions, in which one person reacts to the fixed motions of others, is challenging due to strict interaction constraints and a limited feasible solution space. This paper focuses on a typical scenario: duet dance, where high-quality data is scarce, motion patterns are complex, and the details of human interactions are both intricate and abundant. To tackle these challenges, we propose a novel two-stage framework. In the first stage, we introduce a motion VQ-VAE with separate body-part encoders and a joint decoder, enabling specialized codebooks to enhance representation capacity while dynamically modeling dependencies across body parts during decoding, thereby preventing inconsistencies in the generated motions. In the second stage, we propose a contact-aware diffusion model for reactive motion generation that jointly generates motion and a contact matrix between individuals, enabling explicit interaction modeling and providing guidance toward more precise and constrained interaction dynamics during sampling. Experiments show that our method outperforms Duolando with lower $\text{FID}_k$ (8.89 vs. 25.30) and $\text{FID}_{cd}$ (8.01 vs. 9.97), as well as a higher BED (0.4606 vs. 0.2858), indicating improved interaction fidelity and rhythmic synchronization.