InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint

TL;DR

InterControl is a novel zero-shot motion synthesis framework that enables the generation of multi-human interactions with controllable joint distances, flexible to any number of characters, and guided by language models and inverse kinematics.

Contribution

We propose a flexible, zero-shot motion generation method for multi-human interactions that does not require training on multi-character datasets, utilizing joint pair control and language model guidance.

Findings

Successfully generates multi-human interactions with arbitrary group sizes.

Maintains desired joint distances in synthesized motions.

Works effectively with physics-based character simulators.

Abstract

Text-conditioned motion synthesis has made remarkable progress with the emergence of diffusion models. However, the majority of these motion diffusion models are primarily designed for a single character and overlook multi-human interactions. In our approach, we strive to explore this problem by synthesizing human motion with interactions for a group of characters of any size in a zero-shot manner. The key aspect of our approach is the adaptation of human-wise interactions as pairs of human joints that can be either in contact or separated by a desired distance. In contrast to existing methods that necessitate training motion generation models on multi-human motion datasets with a fixed number of characters, our approach inherently possesses the flexibility to model human interactions involving an arbitrary number of individuals, thereby transcending the limitations imposed by the training data. We introduce a novel controllable motion generation method, InterControl, to encourage the synthesized motions maintaining the desired distance between joint pairs. It consists of a motion controller and an inverse kinematics guidance module that realistically and accurately aligns the joints of synthesized characters to the desired location. Furthermore, we demonstrate that the distance between joint pairs for human-wise interactions can be generated using an off-the-shelf Large Language Model (LLM). Experimental results highlight the capability of our framework to generate interactions with multiple human characters and its potential to work with off-the-shelf physics-based character simulators. Code is available at https://github.com/zhenzhiwang/intercontrol