Good in Bad (GiB): Sifting Through End-user Demonstrations for Learning a Better Policy

TL;DR

GiB is an algorithm that filters out errors in human demonstrations for robot learning, improving policy robustness especially with low-quality data.

Contribution

Introduces GiB, a novel method to automatically identify and discard erroneous parts of demonstrations, enhancing imitation learning from imperfect data.

Findings

GiB improves policy performance in simulated tasks.

GiB enhances real-world robot task success rates.

Filtering with GiB leads to more robust policies.

Abstract

Imitation learning offers a promising framework for enabling robots to acquire diverse skills from human users. However, most imitation learning algorithms assume access to high-quality demonstrations an unrealistic expectation when collecting data from non-expert users, whose demonstrations often contain inadvertent errors. Naively learning from such demonstrations can result in unsafe policy behavior, while discarding entire demonstrations due to occasional mistakes wastes valuable data, especially in low-data settings. In this work, we introduce GiB (Good-in-Bad), an algorithm that automatically identifies and discards erroneous subtasks within demonstrations while preserving high-quality subtasks. The filtered data can then be used by any policy learning algorithm to train more robust policies. GiB first trains a self-supervised model to learn latent features and assigns binary weights to label each demonstration as good or bad. It then models the latent feature distribution of high-quality segments and uses the Mahalanobis distance to detect and evaluate poor-quality subtasks. We validate GiB on the Franka robot in both simulated and real-world multi-step tasks, demonstrating improved policy performance when learning from mixed-quality human demonstrations.