Abstractive Summarization for Low Resource Data using Domain Transfer and Data Synthesis

TL;DR

This paper enhances abstractive summarization for low-resource domains by combining domain transfer and data synthesis, leading to improved ROUGE scores and more coherent summaries in student reflection data.

Contribution

It introduces a combined approach of domain transfer and data synthesis to improve summarization in low-resource settings, demonstrating significant performance gains.

Findings

Tuned models outperform models trained only on target data

Data synthesis further improves ROUGE scores

Combining domain transfer and data synthesis yields the best results

Abstract

Training abstractive summarization models typically requires large amounts of data, which can be a limitation for many domains. In this paper we explore using domain transfer and data synthesis to improve the performance of recent abstractive summarization methods when applied to small corpora of student reflections. First, we explored whether tuning state of the art model trained on newspaper data could boost performance on student reflection data. Evaluations demonstrated that summaries produced by the tuned model achieved higher ROUGE scores compared to model trained on just student reflection data or just newspaper data. The tuned model also achieved higher scores compared to extractive summarization baselines, and additionally was judged to produce more coherent and readable summaries in human evaluations. Second, we explored whether synthesizing summaries of student data could additionally boost performance. We proposed a template-based model to synthesize new data, which when incorporated into training further increased ROUGE scores. Finally, we showed that combining data synthesis with domain transfer achieved higher ROUGE scores compared to only using one of the two approaches.