CUED_speech at TREC 2020 Podcast Summarisation Track

TL;DR

This paper presents a two-step approach combining sentence filtering with a hierarchical model and fine-tuned BART summarization, achieving top performance in the TREC 2020 Podcast Summarisation Challenge.

Contribution

It introduces a novel ensemble-based method using attention filtering and fine-tuned BART for podcast summarization, winning the TREC 2020 challenge.

Findings

Achieved the highest scores in human and automatic evaluations.

Outperformed baseline models and creator descriptions.

Demonstrated effectiveness of ensemble models in summarization.

Abstract

In this paper, we describe our approach for the Podcast Summarisation challenge in TREC 2020. Given a podcast episode with its transcription, the goal is to generate a summary that captures the most important information in the content. Our approach consists of two steps: (1) Filtering redundant or less informative sentences in the transcription using the attention of a hierarchical model; (2) Applying a state-of-the-art text summarisation system (BART) fine-tuned on the Podcast data using a sequence-level reward function. Furthermore, we perform ensembles of three and nine models for our submission runs. We also fine-tune the BART model on the Podcast data as our baseline. The human evaluation by NIST shows that our best submission achieves 1.777 in the EGFB scale, while the score of creator-provided description is 1.291. Our system won the Spotify Podcast Summarisation Challenge in the TREC2020 Podcast Track in both human and automatic evaluation.