Quantifying and extending the coverage of spatial categorization data sets

TL;DR

This paper explores using large language models to generate and extend spatial categorization datasets, demonstrating improved coverage and scalability for multilingual spatial data collection.

Contribution

It introduces a method to leverage LLMs for augmenting spatial datasets, specifically extending the TRPS with new scenes and languages, surpassing previous coverage.

Findings

LLMs produce labels aligning well with human annotations

Adding 42 new scenes improves dataset coverage

The approach scales to multiple languages and larger datasets

Abstract

Variation in spatial categorization across languages is often studied by eliciting human labels for the relations depicted in a set of scenes known as the Topological Relations Picture Series (TRPS). We demonstrate that labels generated by large language models (LLMs) align relatively well with human labels, and show how LLM-generated labels can help to decide which scenes and languages to add to existing spatial data sets. To illustrate our approach we extend the TRPS by adding 42 new scenes, and show that this extension achieves better coverage of the space of possible scenes than two previous extensions of the TRPS. Our results provide a foundation for scaling towards spatial data sets with dozens of languages and hundreds of scenes.