Flying in Clutter on Monocular RGB by Learning in 3D Radiance Fields with Domain Adaptation

TL;DR

This paper presents a method for enabling monocular RGB-based drone navigation in cluttered environments by training in photorealistic 3D simulation and applying domain adaptation to transfer policies to real-world scenarios.

Contribution

The authors introduce a framework combining high-fidelity 3D Gaussian Splatting simulation with adversarial domain adaptation to bridge the sim-to-real gap for monocular RGB navigation.

Findings

Zero-shot transfer enables real-world flight in cluttered environments.

The approach achieves robust navigation under varying illumination conditions.

The method reduces perception gap between simulation and real-world data.

Abstract

Modern autonomous navigation systems predominantly rely on lidar and depth cameras. However, a fundamental question remains: Can flying robots navigate in clutter using solely monocular RGB images? Given the prohibitive costs of real-world data collection, learning policies in simulation offers a promising path. Yet, deploying such policies directly in the physical world is hindered by the significant sim-to-real perception gap. Thus, we propose a framework that couples the photorealism of 3D Gaussian Splatting (3DGS) environments with Adversarial Domain Adaptation. By training in high-fidelity simulation while explicitly minimizing feature discrepancy, our method ensures the policy relies on domain-invariant cues. Experimental results demonstrate that our policy achieves robust zero-shot transfer to the physical world, enabling safe and agile flight in unstructured environments with varying illumination.