Investigating the Role of Centering Theory in the Context of Neural Coreference Resolution Systems

TL;DR

This paper explores how centering theory relates to neural coreference resolution systems, analyzing their adherence to linguistic coherence principles and the impact of contextual embeddings on coreference modeling.

Contribution

It operationalizes centering theory for neural coreference systems and introduces a recency-enhanced version that better captures coreference information.

Findings

Neural coreference resolvers show positive dependence on centering theory.

High-quality neural models benefit little from explicit centering modeling.

Contextualized embeddings encode much coherence information.

Abstract

Centering theory (CT; Grosz et al., 1995) provides a linguistic analysis of the structure of discourse. According to the theory, local coherence of discourse arises from the manner and extent to which successive utterances make reference to the same entities. In this paper, we investigate the connection between centering theory and modern coreference resolution systems. We provide an operationalization of centering and systematically investigate if neural coreference resolvers adhere to the rules of centering theory by defining various discourse metrics and developing a search-based methodology. Our information-theoretic analysis reveals a positive dependence between coreference and centering; but also shows that high-quality neural coreference resolvers may not benefit much from explicitly modeling centering ideas. Our analysis further shows that contextualized embeddings contain much of the coherence information, which helps explain why CT can only provide little gains to modern neural coreference resolvers which make use of pretrained representations. Finally, we discuss factors that contribute to coreference which are not modeled by CT such as world knowledge and recency bias. We formulate a version of CT that also models recency and show that it captures coreference information better compared to vanilla CT.