Extraneousness-Aware Imitation Learning

TL;DR

This paper introduces Extraneousness-Aware Imitation Learning (EIL), a self-supervised method that effectively filters out irrelevant segments in demonstrations to improve visuomotor policy learning from noisy, real-world data.

Contribution

EIL is a novel self-supervised approach that identifies and excludes extraneous, task-irrelevant segments in demonstrations, enhancing imitation learning performance.

Findings

EIL outperforms strong baselines in simulated tasks.

EIL achieves comparable performance to perfect demonstrations in real-world tasks.

EIL effectively filters extraneous segments, improving policy learning.

Abstract

Visual imitation learning provides an effective framework to learn skills from demonstrations. However, the quality of the provided demonstrations usually significantly affects the ability of an agent to acquire desired skills. Therefore, the standard visual imitation learning assumes near-optimal demonstrations, which are expensive or sometimes prohibitive to collect. Previous works propose to learn from noisy demonstrations; however, the noise is usually assumed to follow a context-independent distribution such as a uniform or gaussian distribution. In this paper, we consider another crucial yet underexplored setting -- imitation learning with task-irrelevant yet locally consistent segments in the demonstrations (e.g., wiping sweat while cutting potatoes in a cooking tutorial). We argue that such noise is common in real world data and term them "extraneous" segments. To tackle this problem, we introduce Extraneousness-Aware Imitation Learning (EIL), a self-supervised approach that learns visuomotor policies from third-person demonstrations with extraneous subsequences. EIL learns action-conditioned observation embeddings in a self-supervised manner and retrieves task-relevant observations across visual demonstrations while excluding the extraneous ones. Experimental results show that EIL outperforms strong baselines and achieves comparable policies to those trained with perfect demonstration on both simulated and real-world robot control tasks. The project page can be found at https://sites.google.com/view/eil-website.