Contextualizing Generated Citation Texts

TL;DR

This paper improves abstractive citation text generation by training models to generate the entire context window including the citation, leading to more contextually relevant and human-preferred citations.

Contribution

It introduces a simple modification to generate the full context window, enhancing contextual relevance in citation generation models.

Findings

Generated citations are more contextually relevant.

Human readers prefer the new training approach.

Model effectively uses contextual clues.

Abstract

Abstractive citation text generation is usually framed as an infilling task, where a sequence-to-sequence model is trained to generate a citation given a reference paper and the context window around the target; the generated citation should be a brief discussion of the reference paper as it relates to the citing context. However, examining a recent LED-based citation generation system, we find that many of the generated citations are generic summaries of the reference papers main contribution, ignoring the citation contexts focus on a different topic. To address this problem, we propose a simple modification to the citation text generation task: the generation target is not only the citation itself, but the entire context window, including the target citation. This approach can be easily applied to any abstractive citation generation system, and our experimental results show that training in this way is preferred by human readers and allows the generation model to make use of contextual clues about what topic to discuss and what stance to take.