Spotify at TREC 2020: Genre-Aware Abstractive Podcast Summarization

TL;DR

This paper presents genre-aware abstractive summarization models for podcasts, improving summary relevance and quality by considering genre and named entities, evaluated through human assessments in the TREC 2020 challenge.

Contribution

The paper introduces two supervised abstractive summarization models that incorporate genre and named entities to produce more appropriate podcast summaries.

Findings

Best model outperforms baseline by 9% in quality score

Models effectively incorporate genre and entities for better summaries

Achieved an aggregate quality score of 1.58 compared to 1.49 baseline

Abstract

This paper contains the description of our submissions to the summarization task of the Podcast Track in TREC (the Text REtrieval Conference) 2020. The goal of this challenge was to generate short, informative summaries that contain the key information present in a podcast episode using automatically generated transcripts of the podcast audio. Since podcasts vary with respect to their genre, topic, and granularity of information, we propose two summarization models that explicitly take genre and named entities into consideration in order to generate summaries appropriate to the style of the podcasts. Our models are abstractive, and supervised using creator-provided descriptions as ground truth summaries. The results of the submitted summaries show that our best model achieves an aggregate quality score of 1.58 in comparison to the creator descriptions and a baseline abstractive system which both score 1.49 (an improvement of 9%) as assessed by human evaluators.