Deep Q-Learning-Based Gain Scheduling for Nonlinear Quadcopter Dynamics

TL;DR

This paper introduces a deep Q-network-based gain scheduling method for nonlinear quadcopters, enabling adaptive, stable, and efficient trajectory tracking by selecting from pre-certified gains within a reinforcement learning framework.

Contribution

It proposes a structured reinforcement learning approach that uses pre-certified gain sets and exploits system symmetry to improve quadcopter control stability and performance.

Findings

Accurate trajectory tracking demonstrated in simulations.

Stable attitude control with bounded excursions.

Smooth transition to hover after trajectory completion.

Abstract

This paper presents a deep Q-network (DQN)-based gain-scheduling framework for safety-critical quadcopter trajectory tracking. Instead of directly learning control inputs, the proposed approach selects from a finite set of pre-certified stabilizing gain vectors, enabling reinforcement learning to operate within a structured and stability-preserving control architecture. By exploiting the isotropic structure of the translational dynamics, feedback gains are shared across spatial axes to reduce dimensionality while preserving performance. The learned policy adapts feedback aggressiveness in real time, applying high authority during large transients and reducing gains near convergence to limit control effort. Simulation results using a high-fidelity nonlinear quadcopter model demonstrate accurate trajectory tracking, bounded attitude excursions, smooth transition to hover after the final time, and consistent reward improvement, validating the effectiveness and robustness of the proposed learning-based gain scheduling strategy.