Offline Discovery of Interpretable Skills from Multi-Task Trajectories

TL;DR

LOKI is an end-to-end framework for offline hierarchical skill discovery from multi-task data, combining weakly supervised segmentation, self-supervised refinement, and hierarchical policy learning, achieving high success and meaningful skills.

Contribution

We introduce LOKI, a novel three-stage method for discovering interpretable skills from offline multi-task data without explicit annotations.

Findings

LOKI outperforms standard HIL baselines on D4RL Kitchen benchmark.

Discovered skills are semantically meaningful and align with human intuition.

Skills exhibit compositionality, enabling solving unseen tasks.

Abstract

Hierarchical Imitation Learning is a powerful paradigm for acquiring complex robot behaviors from demonstrations. A central challenge, however, lies in discovering reusable skills from long-horizon, multi-task offline data, especially when the data lacks explicit rewards or subtask annotations. In this work, we introduce LOKI, a three-stage end-to-end learning framework designed for offline skill discovery and hierarchical imitation. The framework commences with a two-stage, weakly supervised skill discovery process: Stage one performs coarse, task-aware macro-segmentation by employing an alignment-enforced Vector Quantized VAE guided by weak task labels. Stage two then refines these segments at a micro-level using a self-supervised sequential model, followed by an iterative clustering process to consolidate skill boundaries. The third stage then leverages these precise boundaries to construct a hierarchical policy within an option-based framework-complete with a learned termination condition beta for explicit skill switching. LOKI achieves high success rates on the challenging D4RL Kitchen benchmark and outperforms standard HIL baselines. Furthermore, we demonstrate that the discovered skills are semantically meaningful, aligning with human intuition, and exhibit compositionality by successfully sequencing them to solve a novel, unseen task.