From Spatial Relations to Spatial Configurations

TL;DR

This paper introduces an extended spatial relation language integrated with AMR to better capture complex spatial configurations in natural language, images, and videos, enhancing reasoning capabilities.

Contribution

It extends existing spatial representation languages with comprehensive, compositional semantics and integrates them into AMR, enabling detailed spatial reasoning in language understanding.

Findings

Annotated diverse datasets with the extended spatial language

Demonstrated improved coverage of spatial concepts

Showed seamless integration with AMR for complex sentences

Abstract

Spatial Reasoning from language is essential for natural language understanding. Supporting it requires a representation scheme that can capture spatial phenomena encountered in language as well as in images and videos. Existing spatial representations are not sufficient for describing spatial configurations used in complex tasks. This paper extends the capabilities of existing spatial representation languages and increases coverage of the semantic aspects that are needed to ground the spatial meaning of natural language text in the world. Our spatial relation language is able to represent a large, comprehensive set of spatial concepts crucial for reasoning and is designed to support the composition of static and dynamic spatial configurations. We integrate this language with the Abstract Meaning Representation(AMR) annotation schema and present a corpus annotated by this extended AMR. To exhibit the applicability of our representation scheme, we annotate text taken from diverse datasets and show how we extend the capabilities of existing spatial representation languages with the fine-grained decomposition of semantics and blend it seamlessly with AMRs of sentences and discourse representations as a whole.