Simulation and Retargeting of Complex Multi-Character Interactions

TL;DR

This paper introduces a deep reinforcement learning method for reproducing and retargeting complex multi-character interactions in physically simulated humanoids, ensuring realistic motion and interaction preservation.

Contribution

It presents a novel reward formulation based on interaction graphs, enabling efficient imitation and retargeting of complex multi-character interactions.

Findings

Successfully reproduces a range of interactions from simple to complex.

Retargets motion to characters with different sizes and morphologies.

Produces physically plausible interactions from motion capture data.

Abstract

We present a method for reproducing complex multi-character interactions for physically simulated humanoid characters using deep reinforcement learning. Our method learns control policies for characters that imitate not only individual motions, but also the interactions between characters, while maintaining balance and matching the complexity of reference data. Our approach uses a novel reward formulation based on an interaction graph that measures distances between pairs of interaction landmarks. This reward encourages control policies to efficiently imitate the character's motion while preserving the spatial relationships of the interactions in the reference motion. We evaluate our method on a variety of activities, from simple interactions such as a high-five greeting to more complex interactions such as gymnastic exercises, Salsa dancing, and box carrying and throwing. This approach can be used to ``clean-up'' existing motion capture data to produce physically plausible interactions or to retarget motion to new characters with different sizes, kinematics or morphologies while maintaining the interactions in the original data.