Adversarial Skill Networks: Unsupervised Robot Skill Learning from Video

TL;DR

This paper introduces a novel adversarial skill learning framework that derives transferable, task-agnostic skill embeddings from unlabeled multi-view videos, enabling robots to learn and reuse skills across diverse tasks without explicit rewards.

Contribution

The paper proposes an unsupervised, adversarial approach to learn a general skill embedding space from unlabeled videos, improving skill reuse and transferability in reinforcement learning.

Findings

Effective skill transfer demonstrated in both simulation and real-world data.

Enables training of control policies for new tasks through skill interpolation.

Outperforms baseline methods in learning transferable skills from unlabeled videos.

Abstract

Key challenges for the deployment of reinforcement learning (RL) agents in the real world are the discovery, representation and reuse of skills in the absence of a reward function. To this end, we propose a novel approach to learn a task-agnostic skill embedding space from unlabeled multi-view videos. Our method learns a general skill embedding independently from the task context by using an adversarial loss. We combine a metric learning loss, which utilizes temporal video coherence to learn a state representation, with an entropy regularized adversarial skill-transfer loss. The metric learning loss learns a disentangled representation by attracting simultaneous viewpoints of the same observations and repelling visually similar frames from temporal neighbors. The adversarial skill-transfer loss enhances re-usability of learned skill embeddings over multiple task domains. We show that the learned embedding enables training of continuous control policies to solve novel tasks that require the interpolation of previously seen skills. Our extensive evaluation with both simulation and real world data demonstrates the effectiveness of our method in learning transferable skills from unlabeled interaction videos and composing them for new tasks. Code, pretrained models and dataset are available at http://robotskills.cs.uni-freiburg.de