Dynamic Full-body Motion Agent with Object Interaction via Blending Pre-trained Modular Controllers

TL;DR

This paper introduces a novel framework for generating long-term, dynamic human-object interaction motions by combining pretrained motion priors and imitation agents, improving success rates and efficiency.

Contribution

It proposes a method that blends pretrained motion models and imitation agents to produce complex, long-term HOI motions, addressing limitations of static datasets and short-term contact models.

Findings

Improved success rates in dynamic HOI motion generation.

Effective blending of pretrained experts enhances performance.

Reduces training time compared to prior methods.

Abstract

Generating physically plausible dynamic motions of human-object interaction (HOI) remains challenging, mainly due to existing HOI datasets limited to static interactions, and pretrained agents capable of either dynamic full-body motions without objects or static HOI motions. Recent works such as InsActor and CLoSD generate HOI motions in planning and execution stages, are yet limited to either static or short-term contacts e.g. striking. In this work, we propose a framework that fulfills dynamic and long-term interaction motions such as running while holding a table, by combining pretrained motion priors and imitation agents in planning and execution stages. In the planning stage, we augment HOI datasets with dynamic priors from a pretrained human motion diffusion model, followed by object trajectory generation. This plans dynamic HOI sequences. In the execution stage, a composer network blends actions of pretrained imitation agents specialized either for dynamic human motions or static HOI motions, enabling spatio-temporal composition of their complementary skills. Our method over relevant prior-arts consistently improves success rates while maintaining interaction for dynamic HOI tasks. Furthermore, blending pretrained experts with our composer achieves competitive performance in significantly reduced training time. Ablation studies validate the effectiveness of our augmentation and composer blending.