Zero-Shot Generalization from Motion Demonstrations to New Tasks

TL;DR

This paper introduces a novel framework combining dynamical systems and graph search to enable robots to generalize learned motions to new, unseen tasks efficiently and with stability.

Contribution

It formalizes the integration of isolated demonstrations using Gaussian Graphs, enabling generalization through Stitching and Chaining frameworks with proven convergence.

Findings

Successful generalization to new tasks in simulations and real robots

Outperforms baseline methods in task generalization

Provides stable and reactive control from few demonstrations

Abstract

Learning motion policies from expert demonstrations is an essential paradigm in modern robotics. While end-to-end models aim for broad generalization, they require large datasets and computationally heavy inference. Conversely, learning dynamical systems (DS) provides fast, reactive, and provably stable control from very few demonstrations. However, existing DS learning methods typically model isolated tasks and struggle to reuse demonstrations for novel behaviors. In this work, we formalize the problem of combining isolated demonstrations within a shared workspace to enable generalization to unseen tasks. The Gaussian Graph is introduced, which reinterprets spatial components of learned motion primitives as discrete vertices with connections to one another. This formulation allows us to bridge continuous control with discrete graph search. We propose two frameworks leveraging this graph: Stitching, for constructing time-invariant DSs, and Chaining, giving a sequence-based DS for complex motions while retaining convergence guarantees. Simulations and real-robot experiments show that these methods successfully generalize to new tasks where baseline methods fail.