InstructCMP: Length Control in Sentence Compression through Instruction-based Large Language Models

TL;DR

InstructCMP leverages instruction-based large language models to enable length-controlled sentence compression, addressing limitations of traditional models by using length priming and instruction fine-tuning for improved performance.

Contribution

This work introduces a novel instruction-based approach for length-controlled sentence compression using LLMs, including new datasets and a length priming technique that enhances zero-shot and fine-tuned performance.

Findings

Length priming improves length control accuracy.

Instruction fine-tuning further enhances performance.

No model modifications are needed for length control.

Abstract

Extractive summarization can produce faithful summaries but often requires additional constraints such as a desired summary length. Traditional sentence compression models do not typically consider the constraints because of their restricted model abilities, which require model modifications for coping with them. To bridge this gap, we propose Instruction-based Compression (InstructCMP), an approach to the sentence compression task that can consider the length constraint through instructions by leveraging the zero-shot task-solving abilities of Large Language Models (LLMs). For this purpose, we created new evaluation datasets by transforming traditional sentence compression datasets into an instruction format. By using the datasets, we first reveal that the current LLMs still face challenges in accurately controlling the length for a compressed text. To address this issue, we propose an approach named "length priming," that incorporates additional length information into the instructions without external resources. While the length priming effectively works in a zero-shot setting, a training dataset with the instructions would further improve the ability of length control. Thus, we additionally created a training dataset in an instruction format to fine-tune the model on it. Experimental results and analysis show that applying the length priming significantly improves performances of InstructCMP in both zero-shot and fine-tuning settings without the need of any model modifications.