Self-Attention for Incomplete Utterance Rewriting

TL;DR

This paper introduces a novel approach for incomplete utterance rewriting that leverages self-attention weights from transformers to directly extract coreference and omission relationships, improving the generation of complete utterances.

Contribution

The method uniquely utilizes self-attention weight matrices instead of traditional word embeddings to enhance incomplete utterance rewriting.

Findings

Achieved competitive results on public IUR datasets.

Demonstrated effectiveness of self-attention weights for coreference and omission extraction.

Improved accuracy in generating complete utterances.

Abstract

Incomplete utterance rewriting (IUR) has recently become an essential task in NLP, aiming to complement the incomplete utterance with sufficient context information for comprehension. In this paper, we propose a novel method by directly extracting the coreference and omission relationship from the self-attention weight matrix of the transformer instead of word embeddings and edit the original text accordingly to generate the complete utterance. Benefiting from the rich information in the self-attention weight matrix, our method achieved competitive results on public IUR datasets.