PromptSum: Parameter-Efficient Controllable Abstractive Summarization

TL;DR

PromptSum introduces a parameter-efficient method combining prompt tuning and discrete entity prompts to achieve high-quality, controllable abstractive summarization with minimal parameter updates.

Contribution

It presents a novel approach that enhances summarization controllability and efficiency by integrating prompt tuning with discrete entity prompts, outperforming existing methods.

Findings

Achieves competitive ROUGE scores on summarization benchmarks.

Provides strong controllability through entity prompts.

Uses significantly fewer parameters than traditional fine-tuning methods.

Abstract

Prompt tuning (PT), a parameter-efficient technique that only tunes the additional prompt embeddings while keeping the backbone pre-trained language model (PLM) frozen, has shown promising results in language understanding tasks, especially in low-resource scenarios. However, effective prompt design methods suitable for generation tasks such as summarization are still lacking. At the same time, summarization guided through instructions (discrete prompts) can achieve a desirable double objective of high quality and controllability in summary generation. Towards a goal of strong summarization performance under the triple conditions of parameter-efficiency, data-efficiency, and controllability, we introduce PromptSum, a method combining PT with a multi-task objective and discrete entity prompts for abstractive summarization. Our model achieves competitive ROUGE results on popular abstractive summarization benchmarks coupled with a strong level of controllability through entities, all while only tuning several orders of magnitude less parameters.