One Arrow, Many Targets: Probing LLMs for Multi-Attribute Controllable Text Summarization

TL;DR

This paper explores multi-attribute controllable text summarization using large language models, introducing a hierarchical adapter fusion technique to better integrate multiple controllable attributes.

Contribution

It introduces a novel hierarchical adapter fusion method for multi-attribute controllable summarization with large language models, addressing a key research gap.

Findings

Hierarchical adapter fusion improves attribute control in summarization.

Different adapter strategies vary in effectiveness for preserving controllability.

The study highlights challenges and future directions for MACS with LLMs.

Abstract

Text summarization is a well-established task within the natural language processing (NLP) community. However, the focus on controllable summarization tailored to user requirements is gaining traction only recently. While several efforts explore controllability in text summarization, the investigation of Multi-Attribute Controllable Summarization (MACS) remains limited. This work addresses this gap by examining the MACS task through the lens of large language models (LLMs), using various learning paradigms, particularly low-rank adapters. We experiment with different popular adapter fine-tuning strategies to assess the effectiveness of the resulting models in retaining cues and patterns associated with multiple controllable attributes. Additionally, we propose and evaluate a novel hierarchical adapter fusion technique to integrate learnings from two distinct controllable attributes. Subsquently, we present our findings, discuss the challenges encountered, and suggest potential avenues for advancing the MACS task.