Efficient Knowledge Transfer for Jump-Starting Control Policy Learning of Multirotors through Physics-Aware Neural Architectures

TL;DR

This paper introduces a physics-aware neural control architecture and a library-based initialization scheme that significantly accelerates policy training for multirotor robots by enabling effective cross-embodiment knowledge transfer, reducing training interactions by up to 73.5%.

Contribution

The work presents a novel physics-aware neural control architecture combined with a similarity-based policy initialization method for efficient cross-embodiment transfer in multirotor control policies.

Findings

Achieves state-of-the-art control performance in simulations and real-world tests.

Reduces environment interactions needed for training by up to 73.5%.

Effective transfer across diverse multirotor configurations.

Abstract

Efficiently training control policies for robots is a major challenge that can greatly benefit from utilizing knowledge gained from training similar systems through cross-embodiment knowledge transfer. In this work, we focus on accelerating policy training using a library-based initialization scheme that enables effective knowledge transfer across multirotor configurations. By leveraging a physics-aware neural control architecture that combines a reinforcement learning-based controller and a supervised control allocation network, we enable the reuse of previously trained policies. To this end, we utilize a policy evaluation-based similarity measure that identifies suitable policies for initialization from a library. We demonstrate that this measure correlates with the reduction in environment interactions needed to reach target performance and is therefore suited for initialization. Extensive simulation and real-world experiments confirm that our control architecture achieves state-of-the-art control performance, and that our initialization scheme saves on average up to $73.5\%$ of environment interactions (compared to training a policy from scratch) across diverse quadrotor and hexarotor designs, paving the way for efficient cross-embodiment transfer in reinforcement learning.