Zero-Shot Strategies for Length-Controllable Summarization

TL;DR

This paper investigates the challenges of zero-shot length control in large language models for summarization, evaluates their performance, and proposes practical methods to improve length adherence without fine-tuning.

Contribution

It introduces a set of methods—length approximation, target adjustment, sample filtering, and automated revisions—that significantly enhance zero-shot length control in LLM summarization.

Findings

Substantial improvements in length compliance achieved

Methods maintain or improve summary quality

Reveals biases and variability in LLM length control

Abstract

Large language models (LLMs) struggle with precise length control, particularly in zero-shot settings. We conduct a comprehensive study evaluating LLMs' length control capabilities across multiple measures and propose practical methods to improve controllability. Our experiments with LLaMA 3 reveal stark differences in length adherence across measures and highlight inherent biases of the model. To address these challenges, we introduce a set of methods: length approximation, target adjustment, sample filtering, and automated revisions. By combining these methods, we demonstrate substantial improvements in length compliance while maintaining or enhancing summary quality, providing highly effective zero-shot strategies for precise length control without the need for model fine-tuning or architectural changes. With our work, we not only advance our understanding of LLM behavior in controlled text generation but also pave the way for more reliable and adaptable summarization systems in real-world applications.