UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility

TL;DR

This paper reviews the integration of large language models with UAVs, proposing a roadmap for developing autonomous, agentic UAVs capable of perception, reasoning, and tool use, advancing beyond current limited autonomy.

Contribution

It provides a comprehensive overview of UAV systems and LLM technology, analyzes multimodal data resources, categorizes key tasks, and proposes a roadmap toward agentic UAVs with autonomous capabilities.

Findings

UAVs are increasingly integrated with LLMs for enhanced autonomy.

Multimodal data resources are critical for training UAV-LMM systems.

A roadmap for developing agentic UAVs is outlined.

Abstract

Low-altitude mobility, exemplified by unmanned aerial vehicles (UAVs), has introduced transformative advancements across various domains, like transportation, logistics, and agriculture. Leveraging flexible perspectives and rapid maneuverability, UAVs extend traditional systems' perception and action capabilities, garnering widespread attention from academia and industry. However, current UAV operations primarily depend on human control, with only limited autonomy in simple scenarios, and lack the intelligence and adaptability needed for more complex environments and tasks. The emergence of large language models (LLMs) demonstrates remarkable problem-solving and generalization capabilities, offering a promising pathway for advancing UAV intelligence. This paper explores the integration of LLMs and UAVs, beginning with an overview of UAV systems' fundamental components and functionalities, followed by an overview of the state-of-the-art in LLM technology. Subsequently, it systematically highlights the multimodal data resources available for UAVs, which provide critical support for training and evaluation. Furthermore, it categorizes and analyzes key tasks and application scenarios where UAVs and LLMs converge. Finally, a reference roadmap towards agentic UAVs is proposed, aiming to enable UAVs to achieve agentic intelligence through autonomous perception, memory, reasoning, and tool utilization. Related resources are available at https://github.com/Hub-Tian/UAVs_Meet_LLMs.