Learning and Exploring Motor Skills with Spacetime Bounds

TL;DR

This paper introduces a DRL framework for physics-based character motor skill learning using spacetime bounds, which are loose constraints that improve robustness and learning efficiency, especially for challenging motions.

Contribution

The novel use of loose spacetime bounds as constraints in DRL enhances robustness and enables exploration of challenging motor skills from reference motions.

Findings

Outperforms state-of-the-art tracking-based DRL methods

Improves learning of difficult motion segments

Facilitates style exploration in motor skills

Abstract

Equipping characters with diverse motor skills is the current bottleneck of physics-based character animation. We propose a Deep Reinforcement Learning (DRL) framework that enables physics-based characters to learn and explore motor skills from reference motions. The key insight is to use loose space-time constraints, termed spacetime bounds, to limit the search space in an early termination fashion. As we only rely on the reference to specify loose spacetime bounds, our learning is more robust with respect to low quality references. Moreover, spacetime bounds are hard constraints that improve learning of challenging motion segments, which can be ignored by imitation-only learning. We compare our method with state-of-the-art tracking-based DRL methods. We also show how to guide style exploration within the proposed framework