Using Soft Constraints To Learn Semantic Models Of Descriptions Of Shapes

TL;DR

This paper develops a semantic model using soft constraints to interpret web users' shape descriptions, enabling accurate object identification in a language game with a 72% success rate.

Contribution

It introduces a novel soft constraint-based semantic modeling approach for understanding shape descriptions in a language game context.

Findings

Descriptions enabled 72% correct object guessing

Model effectively grounds word meanings in context

Approach reduces ambiguity in shape descriptions

Abstract

The contribution of this paper is to provide a semantic model (using soft constraints) of the words used by web-users to describe objects in a language game; a game in which one user describes a selected object of those composing the scene, and another user has to guess which object has been described. The given description needs to be non ambiguous and accurate enough to allow other users to guess the described shape correctly. To build these semantic models the descriptions need to be analyzed to extract the syntax and words' classes used. We have modeled the meaning of these descriptions using soft constraints as a way for grounding the meaning. The descriptions generated by the system took into account the context of the object to avoid ambiguous descriptions, and allowed users to guess the described object correctly 72% of the times.