Learning Robust Control Policies for Inverted Pose on Miniature Blimp Robots

TL;DR

This paper introduces a robust learning framework for inverted pose control in miniature blimp robots, combining high-fidelity simulation, deep reinforcement learning, and a sim-to-real transfer strategy to improve reliability and performance.

Contribution

It presents a novel three-stage framework integrating simulation, deep RL, and a mapping layer for robust inverted control of MBRs, addressing complex dynamics and sim-to-real transfer challenges.

Findings

Higher success rate than energy-shaping controller in simulation

Effective real-world inverted pose maintenance demonstrated

Simulation-to-real transfer achieved with a mapping layer

Abstract

The ability to achieve and maintain inverted poses is essential for unlocking the full agility of miniature blimp robots (MBRs). However, developing reliable inverted control strategies for MBRs remains challenging due to their complex and underactuated dynamics. To address this challenge, we propose a novel framework that enables robust control policy learning for inverted pose on MBRs. The proposed framework consists of three core stages. First, a high-fidelity three-dimensional (3D) simulation environment is constructed and calibrated using real-world MBR motion data. Second, a robust inverted control policy is trained in simulation using a modified Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm combined with a domain randomization strategy. Third, a mapping layer is designed to bridge the sim-to-real gap and facilitate real-world deployment of the learned policy. Comprehensive evaluations in the simulation environment demonstrate that the learned policy achieves a higher success rate compared to the energy-shaping controller. Furthermore, experimental results confirm that the learned policy with a mapping layer enables an MBR to achieve and maintain a fully inverted pose in real-world settings.