FlyMirage: A Fully Automated Generation Pipeline for Diverse and Scalable UAV Flight Data via Generative World Model

TL;DR

FlyMirage is an automated pipeline that creates diverse, realistic, and scalable aerial drone datasets using large language models and generative scene synthesis, advancing vision-language navigation research.

Contribution

It introduces a fully automated data generation pipeline combining LLMs and generative world models for scalable aerial VLN dataset creation.

Findings

Generated a large-scale, diverse aerial VLN dataset.

Produced photorealistic 3D scenes with feasible UAV trajectories.

Automated scene exploration and semantic data acquisition.

Abstract

In the field of Vision-Language Navigation (VLN), aerial datasets remain limited in their ability to combine scale, diversity, and realism, often relying on either costly real-world scenes or visually limited simulations. To address these challenges, we introduce FlyMirage, a highly scalable and fully automated data generation pipeline for aerial VLN. Our approach leverages large language models (LLM) as an environment designer to promote scene diversity, paired with a generative world model that instantiates these designs into high-fidelity 3D Gaussian Splatting (3DGS) scenes. To substantially reduce human labor and ensure the feasibility of flight data, FlyMirage automates scene exploration and semantic information acquisition, and further integrates a dynamically feasible planner for uncrewed aerial vehicle (UAV) trajectory generation. Utilizing this toolchain, we generate a large-scale, diverse, and photorealistic aerial VLN dataset, with dynamically feasible flying trajectories, designed to support the development of next-generation embodied navigation models.