MACSum: Controllable Summarization with Mixed Attributes

TL;DR

MACSum introduces a novel human-annotated dataset for mixed-attribute controllable summarization across news and dialogue domains, and proposes effective prompt-based methods, highlighting the challenges and potential of multi-attribute control.

Contribution

The paper presents MACSum, the first dataset for mixed-attribute controllable summarization, and introduces prompt tuning methods that outperform existing approaches.

Findings

Hard prompt tuning achieves the best performance.

Controlling multiple attributes simultaneously remains challenging.

The dataset enables future research in multi-attribute summarization.

Abstract

Controllable summarization allows users to generate customized summaries with specified attributes. However, due to the lack of designated annotations of controlled summaries, existing works have to craft pseudo datasets by adapting generic summarization benchmarks. Furthermore, most research focuses on controlling single attributes individually (e.g., a short summary or a highly abstractive summary) rather than controlling a mix of attributes together (e.g., a short and highly abstractive summary). In this paper, we propose MACSum, the first human-annotated summarization dataset for controlling mixed attributes. It contains source texts from two domains, news articles and dialogues, with human-annotated summaries controlled by five designed attributes (Length, Extractiveness, Specificity, Topic, and Speaker). We propose two simple and effective parameter-efficient approaches for the new task of mixed controllable summarization based on hard prompt tuning and soft prefix tuning. Results and analysis demonstrate that hard prompt models yield the best performance on all metrics and human evaluations. However, mixed-attribute control is still challenging for summarization tasks. Our dataset and code are available at https://github.com/psunlpgroup/MACSum.