AeroScene: Progressive Scene Synthesis for Aerial Robotics

TL;DR

AeroScene is a hierarchical diffusion model that synthesizes realistic 3D scenes for aerial robotics, improving simulation environments and aiding drone navigation tasks.

Contribution

Introduces AeroScene, a novel hierarchical diffusion model for progressive 3D scene synthesis tailored for aerial robotics applications.

Findings

Outperforms prior scene synthesis methods in experiments

Generates over 1,000 physics-ready 3D scenes for simulation

Enhances drone navigation by providing realistic synthetic environments

Abstract

Generative models have shown substantial impact across multiple domains, their potential for scene synthesis remains underexplored in robotics. This gap is more evident in drone simulators, where simulation environments still rely heavily on manual efforts, which are time-consuming to create and difficult to scale. In this work, we introduce AeroScene, a hierarchical diffusion model for progressive 3D scene synthesis. Our approach leverages hierarchy-aware tokenization and multi-branch feature extraction to reason across both global layouts and local details, ensuring physical plausibility and semantic consistency. This makes AeroScene particularly suited for generating realistic scenes for aerial robotics tasks such as navigation, landing, and perching. We demonstrate its effectiveness through extensive experiments on our newly collected dataset and a public benchmark, showing that AeroScene significantly outperforms prior methods. Furthermore, we use AeroScene to generate a large-scale dataset of over 1,000 physics-ready, high fidelity 3D scenes that can be directly integrated into NVIDIA Isaac Sim. Finally, we illustrate the utility of these generated environments on downstream drone navigation tasks. Our code and dataset are publicly available at aioz-ai.github.io/AeroScene/