Bi-Level Reinforcement Learning Control for an Underactuated Blimp via Center-of-Mass Reconfiguration

TL;DR

This paper presents a bi-level reinforcement learning approach for controlling an underactuated blimp with a movable center-of-mass, improving energy efficiency and tracking accuracy through explicit CoM planning and thrust control.

Contribution

It introduces a novel bi-level RL framework that decouples CoM planning from thrust control for an underactuated blimp, supported by a two-stage learning strategy and convergence analysis.

Findings

Outperforms fixed-CoM and PID controllers in accuracy and robustness.

Enables reliable sim-to-real transfer for the blimp control.

Demonstrates effectiveness through extensive simulations and real-world experiments.

Abstract

This paper investigates goal-directed tracking control of underactuated blimps with center-of-mass (CoM) reconfiguration. Unlike conventional overactuated blimp designs that rely on redundant actuation for simplified control, this paper focuses on a compact architecture consisting of two thrusters and a movable internal slider, aiming to improve energy efficiency and payload capacity. This hardware-efficient configuration introduces significant underactuation and strong nonlinear coupling between CoM dynamics and vehicle motion. To address these challenges, this paper proposes a bi-level reinforcement learning framework that explicitly decouples task-level CoM planning from continuous thrust control. The outer policy determines a target-dependent CoM configuration prior to flight, while the inner policy generates thrust commands to track straight-line references. To ensure stable learning, this paper introduces a two-stage learning strategy, supported by a convergence analysis of the resulting bi-level process. Extensive simulations and real-world experiments on a 27-goal evaluation set demonstrate that the proposed method consistently outperforms fixed-CoM baselines and PID-based controllers, achieving higher tracking accuracy, enhanced robustness, and reliable sim-to-real transfer.