Hierarchical Few-Shot Imitation with Skill Transition Models

TL;DR

FIST is a novel algorithm that enables robots to generalize to unseen tasks by learning skill transition models from offline data and using few demonstrations for imitation, improving long-horizon task performance.

Contribution

The paper introduces FIST, a new method that extracts skills from offline data and generalizes to unseen tasks with minimal demonstrations, addressing a key challenge in skill transfer.

Findings

FIST outperforms prior methods in maze navigation tasks.

FIST effectively manipulates unseen objects in robotic experiments.

FIST demonstrates strong generalization to new tasks with few demonstrations.

Abstract

A desirable property of autonomous agents is the ability to both solve long-horizon problems and generalize to unseen tasks. Recent advances in data-driven skill learning have shown that extracting behavioral priors from offline data can enable agents to solve challenging long-horizon tasks with reinforcement learning. However, generalization to tasks unseen during behavioral prior training remains an outstanding challenge. To this end, we present Few-shot Imitation with Skill Transition Models (FIST), an algorithm that extracts skills from offline data and utilizes them to generalize to unseen tasks given a few downstream demonstrations. FIST learns an inverse skill dynamics model, a distance function, and utilizes a semi-parametric approach for imitation. We show that FIST is capable of generalizing to new tasks and substantially outperforms prior baselines in navigation experiments requiring traversing unseen parts of a large maze and 7-DoF robotic arm experiments requiring manipulating previously unseen objects in a kitchen.