Generating Simulations of Motion Events from Verbal Descriptions

TL;DR

This paper presents a computational framework that converts verbal descriptions of motion into 3D simulations, aiding understanding of motion semantics and distinctions between different types of motion verbs.

Contribution

It introduces a novel method for translating linguistic motion descriptions into dynamic 3D simulations using a formal semantic model and Unity, highlighting distinctions in motion types.

Findings

Generated simulations reveal semantic presuppositions of motion verbs.

The model distinguishes path- and manner-motion in simulations.

Implementation maps linguistic interpretations directly into Unity code.

Abstract

In this paper, we describe a computational model for motion events in natural language that maps from linguistic expressions, through a dynamic event interpretation, into three-dimensional temporal simulations in a model. Starting with the model from (Pustejovsky and Moszkowicz, 2011), we analyze motion events using temporally-traced Labelled Transition Systems. We model the distinction between path- and manner-motion in an operational semantics, and further distinguish different types of manner-of-motion verbs in terms of the mereo-topological relations that hold throughout the process of movement. From these representations, we generate minimal models, which are realized as three-dimensional simulations in software developed with the game engine, Unity. The generated simulations act as a conceptual "debugger" for the semantics of different motion verbs: that is, by testing for consistency and informativeness in the model, simulations expose the presuppositions associated with linguistic expressions and their compositions. Because the model generation component is still incomplete, this paper focuses on an implementation which maps directly from linguistic interpretations into the Unity code snippets that create the simulations.