LMPath: Language-Mediated Priors and Path Generation for Aerial Exploration

TL;DR

LMPath introduces a semantic-aware UAV path planning pipeline that leverages language models and satellite imagery to improve search efficiency in large-scale environments.

Contribution

The paper presents LMPath, a novel pipeline combining language models and vision segmentation to generate exploration priors for UAV search missions.

Findings

LMPath-generated paths outperform traditional planning in simulations.

Real UAV tests validate the effectiveness of LMPath in large environments.

Semantic priors significantly reduce search time compared to geometric coverage.

Abstract

Traditional autonomous UAV search missions rely on geometric coverage patterns that ignore the semantic context of the target, leading to significant time waste in large-scale environments. In this paper we present LMPath, a pipeline for generating language-mediated exploration priors for Unmanned Aerial Vehicle (UAV) search missions that leverages semantics. Given a basic geofence and an object of interest prompt, LMPath uses generative language models to determine what regions of the environment should contain that object and a foundation vision model ran over satellite imagery to segment sub-regions that form the exploration prior. This prior can then be used to generate UAV paths with various objectives, such as minimizing the expected time to locate the object of interest, maximizing the probability that the object is found given a limited travel distance, or narrowing down the search space to sub-regions that are most likely to contain the object. To demonstrate it's capabilities, we used LMPath to generate various UAV paths and ran them using a real UAV over large-scale environments. We also ran simulations to demonstrate how paths generated using LMPath outperform traditional path planning approaches for search missions.