ASkDAgger: Active Skill-level Data Aggregation for Interactive Imitation Learning

TL;DR

ASkDAgger introduces a novel framework for interactive imitation learning that leverages novice plans and teacher feedback to reduce queries, improve generalization, and adapt faster in manipulation tasks.

Contribution

The paper proposes the ASkDAgger framework, integrating novice plans with active learning components to enhance data efficiency and domain adaptation in imitation learning.

Findings

Reduces demonstration annotations needed for effective learning.

Improves generalization and adaptation speed in manipulation tasks.

Validates effectiveness through simulation and real-world experiments.

Abstract

Human teaching effort is a significant bottleneck for the broader applicability of interactive imitation learning. To reduce the number of required queries, existing methods employ active learning to query the human teacher only in uncertain, risky, or novel situations. However, during these queries, the novice's planned actions are not utilized despite containing valuable information, such as the novice's capabilities, as well as corresponding uncertainty levels. To this end, we allow the novice to say: "I plan to do this, but I am uncertain." We introduce the Active Skill-level Data Aggregation (ASkDAgger) framework, which leverages teacher feedback on the novice plan in three key ways: (1) S-Aware Gating (SAG): Adjusts the gating threshold to track sensitivity, specificity, or a minimum success rate; (2) Foresight Interactive Experience Replay (FIER), which recasts valid and relabeled novice action plans into demonstrations; and (3) Prioritized Interactive Experience Replay (PIER), which prioritizes replay based on uncertainty, novice success, and demonstration age. Together, these components balance query frequency with failure incidence, reduce the number of required demonstration annotations, improve generalization, and speed up adaptation to changing domains. We validate the effectiveness of ASkDAgger through language-conditioned manipulation tasks in both simulation and real-world environments. Code, data, and videos are available at https://askdagger.github.io.