Plan-and-Write: Structure-Guided Length Control for LLMs without Model Retraining

TL;DR

This paper introduces a prompt engineering method that enables precise length control in large language models without retraining, using structured planning and word counting within prompts, improving length adherence and output quality.

Contribution

The paper presents a novel structure-guided prompting technique for length control in LLMs that does not require model retraining or complex inference tools.

Findings

Significantly improves length fidelity across six state-of-the-art LLMs.

Achieves up to 37.6% improvement in length adherence.

Maintains or enhances output quality compared to standard prompts.

Abstract

Length control in Large Language Models (LLMs) is a crucial but under-addressed challenge, with applications ranging from voice interfaces requiring concise responses to research summaries needing comprehensive outputs. Current approaches to length control, including Regularized DPO, Length-Instruction Fine Tuning, and tool-augmented methods, typically require expensive model retraining or complex inference-time tooling. This paper presents a prompt engineering methodology that enables precise length control without model retraining. Our structure-guided approach implements deliberate planning and word counting mechanisms within the prompt, encouraging the model to carefully track and adhere to specified length constraints. Comprehensive evaluations across six state-of-the-art LLMs demonstrate that our method significantly improves length fidelity for several models compared to standard prompting when applied to document summarization tasks, particularly for shorter-to-medium length constraints. The proposed technique shows varying benefits across different model architectures, with some models demonstrating up to 37.6% improvement in length adherence. Quality evaluations further reveal that our approach maintains or enhances overall output quality compared to standard prompting techniques. Our approach provides an immediately deployable solution for applications requiring precise length control, particularly valuable for production environments where model retraining is impractical or cost-prohibitive.