Weakly-Supervised Reinforcement Learning for Controllable Behavior

TL;DR

This paper introduces a weakly-supervised reinforcement learning framework that automatically identifies meaningful task subspaces, improving exploration efficiency and performance in complex vision-based control environments.

Contribution

It proposes a novel method for disentangling semantically meaningful tasks from nonsensical ones using weak supervision, enhancing RL exploration and representation.

Findings

Significant performance improvements in vision-based control tasks.

Effective disentanglement of meaningful task subspaces.

Enhanced exploration efficiency in complex environments.

Abstract

Reinforcement learning (RL) is a powerful framework for learning to take actions to solve tasks. However, in many settings, an agent must winnow down the inconceivably large space of all possible tasks to the single task that it is currently being asked to solve. Can we instead constrain the space of tasks to those that are semantically meaningful? In this work, we introduce a framework for using weak supervision to automatically disentangle this semantically meaningful subspace of tasks from the enormous space of nonsensical "chaff" tasks. We show that this learned subspace enables efficient exploration and provides a representation that captures distance between states. On a variety of challenging, vision-based continuous control problems, our approach leads to substantial performance gains, particularly as the complexity of the environment grows.