Learning Task-Agnostic Action Spaces for Movement Optimization

TL;DR

This paper introduces a task-agnostic action space learned through exploration data, enhancing movement optimization efficiency for animated characters without requiring reference movements.

Contribution

It presents a novel, generic method for learning low-level control policies that improve movement optimization across various tasks and algorithms without reference data.

Findings

Policy improves optimization efficiency

Trajectories become more robust to disturbances

Wider optima are easier to find

Abstract

We propose a novel method for exploring the dynamics of physically based animated characters, and learning a task-agnostic action space that makes movement optimization easier. Like several previous papers, we parameterize actions as target states, and learn a short-horizon goal-conditioned low-level control policy that drives the agent's state towards the targets. Our novel contribution is that with our exploration data, we are able to learn the low-level policy in a generic manner and without any reference movement data. Trained once for each agent or simulation environment, the policy improves the efficiency of optimizing both trajectories and high-level policies across multiple tasks and optimization algorithms. We also contribute novel visualizations that show how using target states as actions makes optimized trajectories more robust to disturbances; this manifests as wider optima that are easy to find. Due to its simplicity and generality, our proposed approach should provide a building block that can improve a large variety of movement optimization methods and applications.