Natural Language and Spatial Rules

TL;DR

This paper presents a system that integrates natural language descriptions with spatial semantics and shape grammar formalism to generate and interpret 2D shapes based on spatial arrangements described in English.

Contribution

It introduces a novel framework combining spatial semantics representation with shape grammar to parse and generate shapes from natural language descriptions.

Findings

Successfully parsed various natural language shape descriptions

Integrated spatial semantics with shape grammar formalism

Highlighted open challenges at language-perception interface

Abstract

We develop a system that formally represents spatial semantics concepts within natural language descriptions of spatial arrangements. The system builds on a model of spatial semantics representation according to which words in a sentence are assigned spatial roles and the relations among these roles are represented with spatial relations. We combine our system with the shape grammar formalism that uses shape rules to generate languages (sets) of two-dimensional shapes. Our proposed system consists of pairs of shape rules and verbal rules where the verbal rules describe in English the action of the associated shape rule. We present various types of natural language descriptions of shapes that are successfully parsed by our system and we discuss open questions and challenges we see at the interface of language and perception.