How to Utilize Failure Demo Data?: Effective Data Selection for Imitation Learning Using Distribution Differences in Attention Mechanism

TL;DR

This paper introduces a novel attention-based method to effectively utilize failure demonstration data in imitation learning, enhancing robotic task success rates by distinguishing success-failure discrepancies.

Contribution

It proposes a latent representation learning approach integrated with attention mechanisms and a new metric for selecting beneficial failure data during training.

Findings

Improved task success rates with failure data incorporation

The attention discrepancy metric effectively identifies useful failure samples

Method enhances efficiency of robotic demonstration data utilization

Abstract

Imitation learning for robotic tasks has relied primarily on policies trained only on successful demonstrations, although failures are unavoidable during human data collection. Many existing approaches for exploiting failure data require additional data processing or iterative policy updates through autonomous rollouts, making it difficult to directly and stably utilize failure data accumulated during data collection. In this work, we propose a method that learns latent representations of success-failure discrepancies and incorporates them into the attention mechanism. During inference, an appropriate latent mode is selected from the initial observation to improve action stability. Furthermore, we introduce a post-training metric that quantifies the attention discrepancy between each failure sample and successful demonstrations to select failure data. Simulation results show that the proposed method improves task success rates when trained with failure data and that the proposed metric identifies failure samples that are beneficial for learning when combined with successful demonstrations. These results suggest that the proposed method can support more efficient use of collected demonstrations in robotic data collection pipelines.