Domain Aligned Prefix Averaging for Domain Generalization in Abstractive Summarization

TL;DR

This paper introduces DAPA, a lightweight domain generalization method for abstractive summarization that uses prefix averaging based on summary similarity to improve performance across diverse domains.

Contribution

The paper proposes a novel, efficient prefix averaging approach for domain generalization in abstractive summarization, requiring less complex training algorithms.

Findings

DAPA achieves comparable or superior results to baselines on four summarization domains.

The method enables efficient addition of new source domains through weight averaging.

Prefix tuning provides lightweight fine-tuning for domain adaptation.

Abstract

Domain generalization is hitherto an underexplored area applied in abstractive summarization. Moreover, most existing works on domain generalization have sophisticated training algorithms. In this paper, we propose a lightweight, weight averaging based, Domain Aligned Prefix Averaging approach to domain generalization for abstractive summarization. Given a number of source domains, our method first trains a prefix for each one of them. These source prefixes generate summaries for a small number of target domain documents. The similarity of the generated summaries to their corresponding documents is used for calculating weights required to average source prefixes. In DAPA, prefix tuning allows for lightweight finetuning, and weight averaging allows for the computationally efficient addition of new source domains. When evaluated on four diverse summarization domains, DAPA shows comparable or better performance against the baselines, demonstrating the effectiveness of its prefix averaging scheme.