Planning from Observation and Interaction

TL;DR

This paper introduces a planning-based IRL algorithm enabling real-world robot learning from observation and interaction alone, achieving efficient, scratch learning of manipulation tasks without prior data or pre-training.

Contribution

It presents a novel IRL-based world modeling approach that operates solely on observations and interaction, demonstrating effective real-world manipulation learning from scratch.

Findings

Learned world model enables real-time transfer learning in robots.

Achieves manipulation tasks in under an hour without prior data.

Outperforms existing IRL, RL, and BC methods in sample efficiency.

Abstract

Observational learning requires an agent to learn to perform a task by referencing only observations of the performed task. This work investigates the equivalent setting in real-world robot learning where access to hand-designed rewards and demonstrator actions are not assumed. To address this data-constrained setting, this work presents a planning-based Inverse Reinforcement Learning (IRL) algorithm for world modeling from observation and interaction alone. Experiments conducted entirely in the real-world demonstrate that this paradigm is effective for learning image-based manipulation tasks from scratch in under an hour, without assuming prior knowledge, pre-training, or data of any kind beyond task observations. Moreover, this work demonstrates that the learned world model representation is capable of online transfer learning in the real-world from scratch. In comparison to existing approaches, including IRL, RL, and Behavior Cloning (BC), which have more restrictive assumptions, the proposed approach demonstrates significantly greater sample efficiency and success rates, enabling a practical path forward for online world modeling and planning from observation and interaction. Videos and more at: https://uwrobotlearning.github.io/mpail2/.