Latent Prompt Tuning for Text Summarization

TL;DR

This paper introduces Lotus, a unified model for text summarization that learns latent prompts to enable both controlled and uncontrolled summarization, improving quality and allowing user control without requiring control signals during inference.

Contribution

Lotus is the first model to learn latent prompts for both controlled and uncontrolled summarization, enhancing flexibility and performance.

Findings

Lotus outperforms strong uncontrollable models across four datasets.

Generated summaries can be effectively controlled with user-specified tokens.

Uncontrolled mode of Lotus consistently improves summarization quality.

Abstract

Prompts with different control signals (e.g., length, keywords, etc.) can be used to control text summarization. When control signals are available, they can control the properties of generated summaries and potentially improve summarization quality (since more information are given). Unfortunately, control signals are not already available during inference time. In this paper, we propose Lotus (shorthand for Latent Prompt Tuning for Summarization), which is a single model that can be applied in both controlled and uncontrolled (without control signals) modes. During training, Lotus learns latent prompt representations from prompts with gold control signals using a contrastive learning objective. Experiments show Lotus in uncontrolled mode consistently improves upon strong (uncontrollable) summarization models across four different summarization datasets. We also demonstrate generated summaries can be controlled using prompts with user specified control tokens.