Augmenting learning using symmetry in a biologically-inspired domain

TL;DR

This paper explores how symmetry-based data augmentation in a biologically-inspired quadruped domain accelerates learning in reinforcement learning, especially in data-limited scenarios, by leveraging invariances similar to those in natural sciences.

Contribution

It introduces a method to incorporate symmetry-based data augmentation into reinforcement learning for robotics, demonstrating improved learning speed in a quadruped control task.

Findings

Augmented data accelerates learning in a quadruped RL task.

Symmetry-based augmentation is effective in data-limited regimes.

Method can be applied to speed up robotic learning in realistic scenarios.

Abstract

Invariances to translation, rotation and other spatial transformations are a hallmark of the laws of motion, and have widespread use in the natural sciences to reduce the dimensionality of systems of equations. In supervised learning, such as in image classification tasks, rotation, translation and scale invariances are used to augment training datasets. In this work, we use data augmentation in a similar way, exploiting symmetry in the quadruped domain of the DeepMind control suite (Tassa et al. 2018) to add to the trajectories experienced by the actor in the actor-critic algorithm of Abdolmaleki et al. (2018). In a data-limited regime, the agent using a set of experiences augmented through symmetry is able to learn faster. Our approach can be used to inject knowledge of invariances in the domain and task to augment learning in robots, and more generally, to speed up learning in realistic robotics applications.