Efficient Hyperparameter Optimization for Physics-based Character Animation

TL;DR

This paper introduces a curriculum-based multi-fidelity Bayesian optimization method to efficiently tune hyperparameters in physics-based character animation, significantly reducing computational costs and improving performance in DRL-based control tasks.

Contribution

It proposes a novel CMFBO framework that leverages curriculum-based task difficulty as a fidelity criterion for efficient hyperparameter optimization in physics-based character control.

Findings

Achieves at least 5x efficiency gain over existing methods in DeepMimic.

Outperforms state-of-the-art hyperparameter optimization techniques.

Effectively reduces computational costs in DRL-based character control.

Abstract

Physics-based character animation has seen significant advances in recent years with the adoption of Deep Reinforcement Learning (DRL). However, DRL-based learning methods are usually computationally expensive and their performance crucially depends on the choice of hyperparameters. Tuning hyperparameters for these methods often requires repetitive training of control policies, which is even more computationally prohibitive. In this work, we propose a novel Curriculum-based Multi-Fidelity Bayesian Optimization framework (CMFBO) for efficient hyperparameter optimization of DRL-based character control systems. Using curriculum-based task difficulty as fidelity criterion, our method improves searching efficiency by gradually pruning search space through evaluation on easier motor skill tasks. We evaluate our method on two physics-based character control tasks: character morphology optimization and hyperparameter tuning of DeepMimic. Our algorithm significantly outperforms state-of-the-art hyperparameter optimization methods applicable for physics-based character animation. In particular, we show that hyperparameters optimized through our algorithm result in at least 5x efficiency gain comparing to author-released settings in DeepMimic.