Adaptive Outer-Loop Control of Quadrotors via Reinforcement Learning

TL;DR

This paper presents an adaptive outer-loop control architecture for quadrotors that uses reinforcement learning and online disturbance estimation to improve real-world flight robustness under uncertainties.

Contribution

It introduces a Residual Dynamics Predictor and online calibration methods enabling seamless sim-to-real transfer and disturbance adaptation in quadrotor control.

Findings

Significantly improved trajectory tracking under severe uncertainties.

Effective online disturbance estimation using only flight data.

Robust real-world performance demonstrated on a Crazyflie quadrotor.

Abstract

Deep Reinforcement Learning (DRL) for quadrotor flight control typically relies on Domain Randomization (DR) for sim-to-real transfer, resulting in overly conservative policies that struggle with dynamic disturbances. To overcome this, we propose a novel adaptive control architecture that actively perceives and reacts to instantaneous perturbations. First, we train an optimal outer-loop policy, then replace its reliance on ground-truth disturbance data with a Residual Dynamics Predictor (RDP). The RDP estimates the external forces and moments acting on the aircraft in flight online using only the history of states and control actions. For seamless hardware transfer, we introduce a data-efficient linear calibration bridge and an online thrust correction mechanism that align the simulated latent space with reality using mere seconds of flight data. Real-world validations on a Crazyflie micro-quadrotor demonstrate that our adaptive controller significantly outperforms baselines, maintaining precise trajectory tracking under severe uncertainties including mass variations, asymmetric payloads, and dynamic slung loads