Skypilot: Fine-Tuning LLM with Physical Grounding for AAV Coverage Search

TL;DR

Skypilot enhances large language models for autonomous aerial vehicle coverage and search tasks by integrating physical grounding through Monte Carlo tree search, improving decision-making, trajectory feasibility, and inference speed.

Contribution

This work introduces a novel two-stage framework that physically grounds LLMs with MCTS and fine-tunes Qwen3-4B on generated samples, addressing hallucination and reproducibility issues.

Findings

Improved trajectory feasibility and decision accuracy in simulations.

Enhanced inference speed with maintained solution quality.

Validated effectiveness through real-world flight experiments.

Abstract

Autonomous aerial vehicles (AAVs) have played a pivotal role in coverage operations and search missions. Recent advances in large language models (LLMs) offer promising opportunities to augment AAV intelligence. These advances help address complex challenges like area coverage optimization, dynamic path planning, and adaptive decision-making. However, the absence of physical grounding in LLMs leads to hallucination and reproducibility problems in spatial reasoning and decision-making. To tackle these issues, we present Skypilot, an LLM-enhanced two-stage framework that grounds language models in physical reality by integrating monte carlo tree search (MCTS). In the first stage, we introduce a diversified action space that encompasses generate, regenerate, fine-tune, and evaluate operations, coupled with physics-informed reward functions to ensure trajectory feasibility. In the second stage, we fine-tune Qwen3-4B on 23,000 MCTS-generated samples, achieving substantial inference acceleration while maintaining solution quality. Extensive numerical simulations and real-world flight experiments validate the efficiency and superiority of our proposed approach. Detailed information and experimental results are accessible at https://sky-pilot.top.