InterSyn: Interleaved Learning for Dynamic Motion Synthesis in the Wild

TL;DR

InterSyn introduces an interleaved learning framework that effectively synthesizes realistic multi-person interaction motions by jointly modeling solo and interactive behaviors, improving diversity and naturalness over previous methods.

Contribution

The paper proposes a novel interleaved learning approach with modules for joint interaction modeling and coordination refinement, advancing motion synthesis in complex multi-person scenarios.

Findings

Generated motions show higher text-to-motion alignment.

Produced more diverse and natural interaction sequences.

Sets a new benchmark for multi-person motion synthesis.

Abstract

We present Interleaved Learning for Motion Synthesis (InterSyn), a novel framework that targets the generation of realistic interaction motions by learning from integrated motions that consider both solo and multi-person dynamics. Unlike previous methods that treat these components separately, InterSyn employs an interleaved learning strategy to capture the natural, dynamic interactions and nuanced coordination inherent in real-world scenarios. Our framework comprises two key modules: the Interleaved Interaction Synthesis (INS) module, which jointly models solo and interactive behaviors in a unified paradigm from a first-person perspective to support multiple character interactions, and the Relative Coordination Refinement (REC) module, which refines mutual dynamics and ensures synchronized motions among characters. Experimental results show that the motion sequences generated by InterSyn exhibit higher text-to-motion alignment and improved diversity compared with recent methods, setting a new benchmark for robust and natural motion synthesis. Additionally, our code will be open-sourced in the future to promote further research and development in this area.