Controlling Summarization Length Through EOS Token Weighting

TL;DR

This paper introduces a simple, architecture-agnostic method for controlling the length of generated summaries by emphasizing the EOS token during training, compatible with various models and decoding methods.

Contribution

It proposes a novel approach that increases EOS token importance in loss calculation to control summary length without complex model modifications.

Findings

Effective length control across different architectures

Maintains summary quality while adjusting length

Compatible with various decoding algorithms

Abstract

Controlling the length of generated text can be crucial in various text-generation tasks, including summarization. Existing methods often require complex model alterations, limiting compatibility with pre-trained models. We address these limitations by developing a simple approach for controlling the length of automatic text summaries by increasing the importance of correctly predicting the EOS token in the cross-entropy loss computation. The proposed methodology is agnostic to architecture and decoding algorithms and orthogonal to other inference-time techniques to control the generation length. We tested it with encoder-decoder and modern GPT-style LLMs, and show that this method can control generation length, often without affecting the quality of the summary.