Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks

TL;DR

This paper introduces a memory-augmented neural network for long document coreference resolution that tracks only a limited number of entities, achieving linear runtime and competitive accuracy.

Contribution

It proposes a novel bounded memory neural network that efficiently manages entity tracking, reducing memory and runtime requirements for long document coreference tasks.

Findings

Model remains competitive on OntoNotes and LitBank datasets.

Learned memory management strategy outperforms rule-based approaches.

Achieves linear runtime in document length.

Abstract

Long document coreference resolution remains a challenging task due to the large memory and runtime requirements of current models. Recent work doing incremental coreference resolution using just the global representation of entities shows practical benefits but requires keeping all entities in memory, which can be impractical for long documents. We argue that keeping all entities in memory is unnecessary, and we propose a memory-augmented neural network that tracks only a small bounded number of entities at a time, thus guaranteeing a linear runtime in length of document. We show that (a) the model remains competitive with models with high memory and computational requirements on OntoNotes and LitBank, and (b) the model learns an efficient memory management strategy easily outperforming a rule-based strategy.