Physics-Based Motion Imitation with Adversarial Differential Discriminators

TL;DR

This paper introduces an adversarial differential discriminator (ADD) for multi-objective reinforcement learning, enabling physically simulated characters to imitate complex motions without manually tuning reward functions.

Contribution

The paper proposes a novel adversarial optimization method that reduces manual tuning in motion imitation, applicable across diverse skills and behaviors.

Findings

Achieves high-fidelity motion imitation comparable to state-of-the-art methods.

Eliminates the need for manually-designed reward functions.

Demonstrates effectiveness on a variety of acrobatic and agile behaviors.

Abstract

Multi-objective optimization problems, which require the simultaneous optimization of multiple objectives, are prevalent across numerous applications. Existing multi-objective optimization methods often rely on manually-tuned aggregation functions to formulate a joint optimization objective. The performance of such hand-tuned methods is heavily dependent on careful weight selection, a time-consuming and laborious process. These limitations also arise in the setting of reinforcement-learning-based motion tracking methods for physically simulated characters, where intricately crafted reward functions are typically used to achieve high-fidelity results. Such solutions not only require domain expertise and significant manual tuning, but also limit the applicability of the resulting reward function across diverse skills. To bridge this gap, we present a novel adversarial multi-objective optimization technique that is broadly applicable to a range of multi-objective reinforcement-learning tasks, including motion tracking. Our proposed Adversarial Differential Discriminator (ADD) receives a single positive sample, yet is still effective at guiding the optimization process. We demonstrate that our technique can enable characters to closely replicate a variety of acrobatic and agile behaviors, achieving comparable quality to state-of-the-art motion-tracking methods, without relying on manually-designed reward functions. Code and results are available at https://add-moo.github.io/.