Curriculum-Based Reinforcement Learning for Autonomous UAV Navigation in Unknown Curved Tubular Conduits

TL;DR

This paper introduces a reinforcement learning method for drones to navigate unknown curved tubes using only local sensor data, outperforming traditional methods.

Contribution

The novel approach introduces a curriculum-based RL framework with explicit turn negotiation formulation and reward design for tubular navigation.

Findings

The RL agent using PPO outperforms deterministic controllers in navigating curved tubes with limited geometric information.

A turning-negotiation mechanism combining visibility, memory, and LiDAR symmetry ensures stable navigation in partial observability.

The learned behavior is robust and transferable to high-fidelity 3D environments with continuous physical dynamics.

Abstract

Autonomous drone navigation in confined tubular environments remains a major challenge due to the constraining geometry of the conduits, the proximity of the walls, and the perceptual limitations inherent to such scenarios. We propose a reinforcement learning (RL) approach enabling a drone to navigate unknown three-dimensional tubes without any prior knowledge of their geometry, relying solely on local observations from a Light Detection and Ranging (LiDAR) sensor and a conditional visual detection of the tube center. In contrast, the Pure Pursuit algorithm, used as a deterministic baseline, benefits from explicit access to the centerline, creating an information asymmetry designed to assess the ability of RL to compensate for the absence of a geometric model. The agent is trained through a progressive curriculum learning strategy that gradually exposes it to increasingly curved geometries, where the tube center frequently disappears from the visual field. A turning-negotiation mechanism, based on the combination of direct visibility, directional memory, and LiDAR symmetry cues, proves essential for ensuring stable navigation under such partial observability conditions. Experiments show that the Proximal Policy Optimization (PPO) policy acquires robust and generalizable behavior, consistently outperforming the deterministic controller despite its limited access to geometric information. Validation in a high-fidelity three-dimensional environment further confirms the transferability of the learned behavior to continuous physical dynamics. In particular, this work introduces an explicit formulation of the turn negotiation problem in tubular navigation, coupled with a reward design and evaluation metrics that make turn-handling behavior measurable and analyzable. This explicit focus on turn negotiation distinguishes our approach from prior learning-based and heuristic methods. The proposed approach thus provides a complete framework for autonomous navigation in unknown tubular environments and opens perspectives for industrial, underground, or medical applications where progressing through narrow and weakly perceptive conduits represents a central challenge.

Figures