Learning Predictive Models From Observation and Interaction

TL;DR

This paper introduces a method for robots to learn predictive models by combining interaction data with observational videos, enabling skill acquisition without direct robotic demonstrations.

Contribution

It formulates a graphical model that treats actions as observed or unobserved variables and uses domain priors to leverage passive observational data for robotic learning.

Findings

Robots can learn tool use from human videos without robotic demonstrations.

The method effectively integrates observational and interaction data.

Robots successfully perform manipulation tasks after training with diverse datasets.

Abstract

Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works, and then use this learned model to plan coordinated sequences of actions to bring about desired outcomes. However, learning a model that captures the dynamics of complex skills represents a major challenge: if the agent needs a good model to perform these skills, it might never be able to collect the experience on its own that is required to learn these delicate and complex behaviors. Instead, we can imagine augmenting the training set with observational data of other agents, such as humans. Such data is likely more plentiful, but represents a different embodiment. For example, videos of humans might show a robot how to use a tool, but (i) are not annotated with suitable robot actions, and (ii) contain a systematic distributional shift due to the embodiment differences between humans and robots. We address the first challenge by formulating the corresponding graphical model and treating the action as an observed variable for the interaction data and an unobserved variable for the observation data, and the second challenge by using a domain-dependent prior. In addition to interaction data, our method is able to leverage videos of passive observations in a driving dataset and a dataset of robotic manipulation videos. A robotic planning agent equipped with our method can learn to use tools in a tabletop robotic manipulation setting by observing humans without ever seeing a robotic video of tool use.