Following Length Constraints in Instructions

TL;DR

This paper introduces a training method for instruction-following models that allows explicit control over response length, improving performance in length-constrained tasks compared to existing models like GPT-4 and Llama 3.

Contribution

The authors develop a training approach enabling models to adhere to specified length constraints during inference, addressing length bias issues in instruction-following models.

Findings

Models trained with length control outperform standard models in length-constrained evaluations.

The approach mitigates length bias in instruction-following models.

Controlled models show superior adherence to length instructions in tests.

Abstract

Aligned instruction following models can better fulfill user requests than their unaligned counterparts. However, it has been shown that there is a length bias in evaluation of such models, and that training algorithms tend to exploit this bias by learning longer responses. In this work we show how to train models that can be controlled at inference time with instructions containing desired length constraints. Such models are superior in length instructed evaluations, outperforming standard instruction following models such as GPT4, Llama 3 and Mixtral.