Merkel Podcast Corpus: A Multimodal Dataset Compiled from 16 Years of Angela Merkel's Weekly Video Podcasts

TL;DR

The Merkel Podcast Corpus is a comprehensive multimodal dataset of Angela Merkel's weekly video podcasts spanning 16 years, useful for speech and face generation research.

Contribution

This paper presents the first large-scale, multimodal, single-speaker German dataset with audio, visual, and text data, along with a general curation pipeline.

Findings

Demonstrated dataset utility in talking face generation

Applied dataset to text-to-speech synthesis

Provided statistical analyses of the data

Abstract

We introduce the Merkel Podcast Corpus, an audio-visual-text corpus in German collected from 16 years of (almost) weekly Internet podcasts of former German chancellor Angela Merkel. To the best of our knowledge, this is the first single speaker corpus in the German language consisting of audio, visual and text modalities of comparable size and temporal extent. We describe the methods used with which we have collected and edited the data which involves downloading the videos, transcripts and other metadata, forced alignment, performing active speaker recognition and face detection to finally curate the single speaker dataset consisting of utterances spoken by Angela Merkel. The proposed pipeline is general and can be used to curate other datasets of similar nature, such as talk show contents. Through various statistical analyses and applications of the dataset in talking face generation and TTS, we show the utility of the dataset. We argue that it is a valuable contribution to the research community, in particular, due to its realistic and challenging material at the boundary between prepared and spontaneous speech.