Equivariant Reinforcement Learning Frameworks for Quadrotor Low-Level Control

TL;DR

This paper introduces equivariant reinforcement learning frameworks that leverage symmetries in quadrotor dynamics to improve data efficiency and generalization in low-level control, reducing training data needs and enhancing flight performance.

Contribution

It develops novel equivariant RL models that encode rotational and reflectional symmetries, enabling better generalization and efficiency in quadrotor control tasks.

Findings

Equivariant models outperform non-equivariant ones in learning efficiency.

Symmetry encoding improves generalization across configurations.

Experimental results show enhanced flight performance.

Abstract

Improving sampling efficiency and generalization capability is critical for the successful data-driven control of quadrotor unmanned aerial vehicles (UAVs) that are inherently unstable. While various reinforcement learning (RL) approaches have been applied to autonomous quadrotor flight, they often require extensive training data, posing multiple challenges and safety risks in practice. To address these issues, we propose data-efficient, equivariant monolithic and modular RL frameworks for quadrotor low-level control. Specifically, by identifying the rotational and reflectional symmetries in quadrotor dynamics and encoding these symmetries into equivariant network models, we remove redundancies of learning in the state-action space. This approach enables the optimal control action learned in one configuration to automatically generalize into other configurations via symmetry, thereby enhancing data efficiency. Experimental results demonstrate that our equivariant approaches significantly outperform their non-equivariant counterparts in terms of learning efficiency and flight performance.