Efficient RL via Disentangled Environment and Agent Representations

TL;DR

This paper introduces a method for reinforcement learning that leverages disentangled representations of the environment and agent, improving performance across diverse visual simulation tasks by incorporating inexpensive visual knowledge into the learning process.

Contribution

The paper presents a novel structured representation approach for RL that uses visual knowledge of the agent, integrated via an auxiliary loss, to enhance learning efficiency and effectiveness.

Findings

Outperforms state-of-the-art model-free methods on 18 environments

Effective across 5 different robotic simulation scenarios

Utilizes inexpensive visual agent information to improve representations

Abstract

Agents that are aware of the separation between themselves and their environments can leverage this understanding to form effective representations of visual input. We propose an approach for learning such structured representations for RL algorithms, using visual knowledge of the agent, such as its shape or mask, which is often inexpensive to obtain. This is incorporated into the RL objective using a simple auxiliary loss. We show that our method, Structured Environment-Agent Representations, outperforms state-of-the-art model-free approaches over 18 different challenging visual simulation environments spanning 5 different robots. Website at https://sear-rl.github.io/