Text-to-OverpassQL: A Natural Language Interface for Complex Geodata Querying of OpenStreetMap

TL;DR

This paper introduces Text-to-OverpassQL, a natural language interface for querying OpenStreetMap data, including a new dataset, evaluation metrics, and baseline models to facilitate research in this area.

Contribution

It presents the first dataset and evaluation framework for translating natural language to OverpassQL queries, enabling easier access to geospatial data for users.

Findings

Sequence models can generate OverpassQL with moderate accuracy.

Large language models show potential with in-context learning.

Evaluation by executing queries highlights strengths and weaknesses of models.

Abstract

We present Text-to-OverpassQL, a task designed to facilitate a natural language interface for querying geodata from OpenStreetMap (OSM). The Overpass Query Language (OverpassQL) allows users to formulate complex database queries and is widely adopted in the OSM ecosystem. Generating Overpass queries from natural language input serves multiple use-cases. It enables novice users to utilize OverpassQL without prior knowledge, assists experienced users with crafting advanced queries, and enables tool-augmented large language models to access information stored in the OSM database. In order to assess the performance of current sequence generation models on this task, we propose OverpassNL, a dataset of 8,352 queries with corresponding natural language inputs. We further introduce task specific evaluation metrics and ground the evaluation of the Text-to-OverpassQL task by executing the queries against the OSM database. We establish strong baselines by finetuning sequence-to-sequence models and adapting large language models with in-context examples. The detailed evaluation reveals strengths and weaknesses of the considered learning strategies, laying the foundations for further research into the Text-to-OverpassQL task.