Disentangled Hierarchical VAE for 3D Human-Human Interaction Generation

TL;DR

This paper introduces DHVAE, a hierarchical VAE with disentangled latent spaces and diffusion-based synthesis, to generate realistic, physically plausible 3D human interactions with improved fidelity and control.

Contribution

It proposes a novel disentangled hierarchical VAE with a diffusion process and contrastive learning to enhance 3D human interaction generation.

Findings

Achieves superior motion fidelity and physical plausibility.

Improves computational efficiency in interaction synthesis.

Enhances control over interaction semantics.

Abstract

Generating realistic 3D Human-Human Interaction (HHI) requires coherent modeling of the physical plausibility of the agents and their interaction semantics. Existing methods compress all motion information into a single latent representation, limiting their ability to capture fine-grained actions and inter-agent interactions. This often leads to semantic misalignment and physically implausible artifacts, such as penetration or missed contact. We propose Disentangled Hierarchical Variational Autoencoder (DHVAE) based latent diffusion for structured and controllable HHI generation. DHVAE explicitly disentangles the global interaction context and individual motion patterns into a decoupled latent structure by employing a CoTransformer module. To mitigate implausible and physically inconsistent contacts in HHI, we incorporate contrastive learning constraints with our DHVAE to promote a more discriminative and physically plausible latent interaction space. For high-fidelity interaction synthesis, DHVAE employs a DDIM-based diffusion denoising process in the hierarchical latent space, enhanced by a skip-connected AdaLN-Transformer denoiser. Extensive evaluations show that DHVAE achieves superior motion fidelity, text alignment, and physical plausibility with greater computational efficiency.