DynaMimicGen: A Data Generation Framework for Robot Learning of Dynamic Tasks

TL;DR

DynaMimicGen (D-MG) is a scalable data generation framework that enables robot learning of dynamic manipulation tasks from minimal supervision by generating adaptable, realistic trajectories that support diverse and changing environments.

Contribution

D-MG introduces a novel method for generating dynamic, task-consistent trajectories from limited demonstrations, improving data efficiency and adaptability in robot learning for dynamic tasks.

Findings

Robots trained on D-MG data perform well on complex manipulation benchmarks.

D-MG-generated data enables generalization to dynamic and unpredictable environments.

The framework reduces the need for extensive human demonstrations.

Abstract

Learning robust manipulation policies typically requires large and diverse datasets, the collection of which is time-consuming, labor-intensive, and often impractical for dynamic environments. In this work, we introduce DynaMimicGen (D-MG), a scalable dataset generation framework that enables policy training from minimal human supervision while uniquely supporting dynamic task settings. Given only a few human demonstrations, D-MG first segments the demonstrations into meaningful sub-tasks, then leverages Dynamic Movement Primitives (DMPs) to adapt and generalize the demonstrated behaviors to novel and dynamically changing environments. Improving prior methods that rely on static assumptions or simplistic trajectory interpolation, D-MG produces smooth, realistic, and task-consistent Cartesian trajectories that adapt in real time to changes in object poses, robot states, or scene geometry during task execution. Our method supports different scenarios - including scene layouts, object instances, and robot configurations - making it suitable for both static and highly dynamic manipulation tasks. We show that robot agents trained via imitation learning on D-MG-generated data achieve strong performance across long-horizon and contact-rich benchmarks, including tasks like cube stacking and placing mugs in drawers, even under unpredictable environment changes. By eliminating the need for extensive human demonstrations and enabling generalization in dynamic settings, D-MG offers a powerful and efficient alternative to manual data collection, paving the way toward scalable, autonomous robot learning.