Adversarial Feature Training for Generalizable Robotic Visuomotor Control

TL;DR

This paper introduces an adversarial training method that enables reinforcement learning-based visuomotor policies to generalize across different task domains in robotics, reducing the need for extensive data collection.

Contribution

It presents a novel adversarial domain transfer approach that allows RL-trained visuomotor policies to adapt to new environments using only still images.

Findings

Outperforms prior methods in robotic picking and pouring tasks

Enables policy transfer with minimal data from target domain

Demonstrates improved generalization to novel objects and viewpoints

Abstract

Deep reinforcement learning (RL) has enabled training action-selection policies, end-to-end, by learning a function which maps image pixels to action outputs. However, it's application to visuomotor robotic policy training has been limited because of the challenge of large-scale data collection when working with physical hardware. A suitable visuomotor policy should perform well not just for the task-setup it has been trained for, but also for all varieties of the task, including novel objects at different viewpoints surrounded by task-irrelevant objects. However, it is impractical for a robotic setup to sufficiently collect interactive samples in a RL framework to generalize well to novel aspects of a task. In this work, we demonstrate that by using adversarial training for domain transfer, it is possible to train visuomotor policies based on RL frameworks, and then transfer the acquired policy to other novel task domains. We propose to leverage the deep RL capabilities to learn complex visuomotor skills for uncomplicated task setups, and then exploit transfer learning to generalize to new task domains provided only still images of the task in the target domain. We evaluate our method on two real robotic tasks, picking and pouring, and compare it to a number of prior works, demonstrating its superiority.