Affordance Learning from Play for Sample-Efficient Policy Learning

TL;DR

This paper introduces VAPO, a method that combines visual affordance models learned from human play data with reinforcement learning to enable robots to learn manipulation policies more efficiently and generalize better to new objects.

Contribution

The paper presents a novel approach that integrates self-supervised affordance learning with policy optimization, improving sample efficiency and generalization in robotic manipulation tasks.

Findings

Policies train 4x faster than baselines

Affordance-guided policies generalize better to novel objects

The method is effective in both simulation and real-world experiments

Abstract

Robots operating in human-centered environments should have the ability to understand how objects function: what can be done with each object, where this interaction may occur, and how the object is used to achieve a goal. To this end, we propose a novel approach that extracts a self-supervised visual affordance model from human teleoperated play data and leverages it to enable efficient policy learning and motion planning. We combine model-based planning with model-free deep reinforcement learning (RL) to learn policies that favor the same object regions favored by people, while requiring minimal robot interactions with the environment. We evaluate our algorithm, Visual Affordance-guided Policy Optimization (VAPO), with both diverse simulation manipulation tasks and real world robot tidy-up experiments to demonstrate the effectiveness of our affordance-guided policies. We find that our policies train 4x faster than the baselines and generalize better to novel objects because our visual affordance model can anticipate their affordance regions.