Zero-Shot UAV Navigation in Forests via Relightable 3D Gaussian Splatting

TL;DR

This paper introduces a novel reinforcement learning approach for UAV navigation in forests that leverages relightable 3D scene representations to achieve robust, zero-shot transfer across diverse lighting conditions.

Contribution

It proposes Relightable 3D Gaussian Splatting for scene representation, enabling explicit lighting manipulation and improved policy generalization in outdoor UAV navigation.

Findings

Successful real-world navigation in complex forests at high speeds

Robust performance under drastic lighting variations without fine-tuning

Effective zero-shot transfer from simulation to real-world environments

Abstract

UAV navigation in unstructured outdoor environments using passive monocular vision is hindered by the substantial visual domain gap between simulation and reality. While 3D Gaussian Splatting enables photorealistic scene reconstruction from real-world data, existing methods inherently couple static lighting with geometry, severely limiting policy generalization to dynamic real-world illumination. In this paper, we propose a novel end-to-end reinforcement learning framework designed for effective zero-shot transfer to unstructured outdoors. Within a high-fidelity simulation grounded in real-world data, our policy is trained to map raw monocular RGB observations directly to continuous control commands. To overcome photometric limitations, we introduce Relightable 3D Gaussian Splatting, which decomposes scene components to enable explicit, physically grounded editing of environmental lighting within the neural representation. By augmenting training with diverse synthesized lighting conditions ranging from strong directional sunlight to diffuse overcast skies, we compel the policy to learn robust, illumination-invariant visual features. Extensive real-world experiments demonstrate that a lightweight quadrotor achieves robust, collision-free navigation in complex forest environments at speeds up to 10 m/s, exhibiting significant resilience to drastic lighting variations without fine-tuning.