Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogames

TL;DR

This paper demonstrates that training models on demonstrations from 'blindfolded' experts, who have limited task information, leads to better generalization to new tasks in robotics and videogames, supported by theoretical analysis.

Contribution

It introduces the concept of 'blindfolded' experts for behavioral cloning and shows that this approach improves generalization, backed by theoretical insights and real-world experiments.

Findings

Blindfolded expert demonstrations lead to better generalization.

Theoretical analysis confirms error scales with a0b7I/m.

Experiments on robot and videogame tasks support the approach.

Abstract

Behavioral cloning is a simple yet effective technique for learning sequential decision-making from demonstrations. Recently, it has gained prominence as the core of foundation models for the physical world, where achieving generalization requires countless demonstrations of a multitude of tasks. Typically, a human expert with full information on the task demonstrates a (nearly) optimal behavior. In this paper, we propose to hide some of the task's information from the demonstrator. This ``blindfolded'' expert is compelled to employ non-trivial exploration to solve the task. We show that cloning the blindfolded expert generalizes better to unseen tasks than its fully-informed counterpart. We conduct experiments of real-world robot peg insertion tasks with (limited) human demonstrations, alongside videogames from the Procgen benchmark. Additionally, we support our findings with theoretical analysis, which confirms that the generalization error scales with $\sqrt{I/m}$, where $I$ measures the amount of task information available to the demonstrator, and $m$ is the number of demonstrated tasks. Both theory and practice indicate that cloning blindfolded experts generalizes better with fewer demonstrated tasks. Project page with videos and code: https://sites.google.com/view/blindfoldedexperts/home