Controlled Diversity: Length-optimized Natural Language Generation

TL;DR

This paper introduces a method to train large language models to generate outputs of specified lengths, improving their utility in applications with strict length constraints by augmenting data and fine-tuning.

Contribution

It presents a novel approach combining data augmentation and fine-tuning to enable LLMs to better adhere to length requirements during generation.

Findings

Models trained with the proposed method better follow length constraints.

The approach can affect response quality depending on training data.

Training on the model's own responses mitigates quality issues.

Abstract

LLMs are not generally able to adjust the length of their outputs based on strict length requirements, a capability that would improve their usefulness in applications that require adherence to diverse user and system requirements. We present an approach to train LLMs to acquire this capability by augmenting existing data and applying existing fine-tuning techniques, which we compare based on the trained models' adherence to the length requirement and overall response quality relative to the baseline model. Our results demonstrate that these techniques can be successfully applied to train LLMs to adhere to length requirements, with the trained models generating texts which better align to the length requirements. Our results indicate that our method may change the response quality when using training data that was not generated by the baseline model. This allows simultaneous alignment to another training objective in certain scenarios, but is undesirable otherwise. Training on a dataset containing the model's own responses eliminates this issue.