Living a discrete life in a continuous world: Reference with distributed representations

TL;DR

This paper introduces a new neural network architecture with external memory for cross-modal entity tracking, effectively handling both continuous and discrete aspects of reference, and compares its performance with existing models.

Contribution

It proposes a novel neural architecture inspired by DRSs that dynamically manages referents, advancing the modeling of reference in language understanding.

Findings

Outperforms traditional neural networks on the referential task

Underperforms compared to Memory Networks with external memory

Demonstrates potential in handling discrete reference aspects

Abstract

Reference is a crucial property of language that allows us to connect linguistic expressions to the world. Modeling it requires handling both continuous and discrete aspects of meaning. Data-driven models excel at the former, but struggle with the latter, and the reverse is true for symbolic models. This paper (a) introduces a concrete referential task to test both aspects, called cross-modal entity tracking; (b) proposes a neural network architecture that uses external memory to build an entity library inspired in the DRSs of DRT, with a mechanism to dynamically introduce new referents or add information to referents that are already in the library. Our model shows promise: it beats traditional neural network architectures on the task. However, it is still outperformed by Memory Networks, another model with external memory.