Equivariant Reinforcement Learning under Partial Observability

TL;DR

This paper introduces equivariant reinforcement learning agents that leverage symmetries in partially observable environments, leading to more sample-efficient and effective robotic learning in simulation and real-world tasks.

Contribution

It proposes encoding equivariance into neural networks for reinforcement learning, improving sample efficiency and performance in partially observable robotic domains.

Findings

Equivariant agents outperform non-equivariant methods in sample efficiency.

Significant performance improvements demonstrated in robotic simulation tasks.

Effective transfer of solutions across related scenarios using symmetry encoding.

Abstract

Incorporating inductive biases is a promising approach for tackling challenging robot learning domains with sample-efficient solutions. This paper identifies partially observable domains where symmetries can be a useful inductive bias for efficient learning. Specifically, by encoding the equivariance regarding specific group symmetries into the neural networks, our actor-critic reinforcement learning agents can reuse solutions in the past for related scenarios. Consequently, our equivariant agents outperform non-equivariant approaches significantly in terms of sample efficiency and final performance, demonstrated through experiments on a range of robotic tasks in simulation and real hardware.