Can Euclidean Symmetry be Leveraged in Reinforcement Learning and Planning?

TL;DR

This paper explores how Euclidean symmetry can be exploited to improve reinforcement learning and planning algorithms, unifying discrete and continuous symmetry approaches and demonstrating benefits through empirical results.

Contribution

It introduces a unified theory for Euclidean symmetry in RL and planning, extends 2D path planning to continuous MDPs, and proposes equivariant sampling-based planning algorithms.

Findings

Empirical evidence shows benefits of Euclidean equivariance in control tasks.

Unified theory bridges discrete and continuous symmetry in RL and planning.

Proposed algorithms improve efficiency in natural control problems.

Abstract

In robotic tasks, changes in reference frames typically do not influence the underlying physical properties of the system, which has been known as invariance of physical laws.These changes, which preserve distance, encompass isometric transformations such as translations, rotations, and reflections, collectively known as the Euclidean group. In this work, we delve into the design of improved learning algorithms for reinforcement learning and planning tasks that possess Euclidean group symmetry. We put forth a theory on that unify prior work on discrete and continuous symmetry in reinforcement learning, planning, and optimal control. Algorithm side, we further extend the 2D path planning with value-based planning to continuous MDPs and propose a pipeline for constructing equivariant sampling-based planning algorithms. Our work is substantiated with empirical evidence and illustrated through examples that explain the benefits of equivariance to Euclidean symmetry in tackling natural control problems.