IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data

TL;DR

IRIS is a scalable offline learning framework for robot manipulation that combines goal-conditioned control with goal selection to improve performance on diverse, large datasets.

Contribution

IRIS introduces a novel two-level approach that separates goal setting from low-level control, enabling effective learning from large, diverse, and suboptimal demonstration datasets.

Findings

IRIS achieves successful policy learning on large-scale datasets.

IRIS outperforms baseline methods on multiple datasets.

IRIS effectively handles suboptimal and diverse demonstrations.

Abstract

Learning from offline task demonstrations is a problem of great interest in robotics. For simple short-horizon manipulation tasks with modest variation in task instances, offline learning from a small set of demonstrations can produce controllers that successfully solve the task. However, leveraging a fixed batch of data can be problematic for larger datasets and longer-horizon tasks with greater variations. The data can exhibit substantial diversity and consist of suboptimal solution approaches. In this paper, we propose Implicit Reinforcement without Interaction at Scale (IRIS), a novel framework for learning from large-scale demonstration datasets. IRIS factorizes the control problem into a goal-conditioned low-level controller that imitates short demonstration sequences and a high-level goal selection mechanism that sets goals for the low-level and selectively combines parts of suboptimal solutions leading to more successful task completions. We evaluate IRIS across three datasets, including the RoboTurk Cans dataset collected by humans via crowdsourcing, and show that performant policies can be learned from purely offline learning. Additional results at https://sites.google.com/stanford.edu/iris/ .