Topical Segmentation of Spoken Narratives: A Test Case on Holocaust Survivor Testimonies

TL;DR

This paper addresses the challenge of segmenting spoken narratives, specifically Holocaust survivor testimonies, by developing and testing new algorithms that outperform previous methods in identifying topic boundaries.

Contribution

It introduces novel segmentation algorithms tailored for unstructured spoken narratives and demonstrates their effectiveness on Holocaust testimonies, a previously underexplored domain.

Findings

Algorithms outperform previous methods in boundary detection

Mutual information is effective for identifying segment boundaries

Holocaust testimonies serve as a valuable test case for segmentation techniques

Abstract

The task of topical segmentation is well studied, but previous work has mostly addressed it in the context of structured, well-defined segments, such as segmentation into paragraphs, chapters, or segmenting text that originated from multiple sources. We tackle the task of segmenting running (spoken) narratives, which poses hitherto unaddressed challenges. As a test case, we address Holocaust survivor testimonies, given in English. Other than the importance of studying these testimonies for Holocaust research, we argue that they provide an interesting test case for topical segmentation, due to their unstructured surface level, relative abundance (tens of thousands of such testimonies were collected), and the relatively confined domain that they cover. We hypothesize that boundary points between segments correspond to low mutual information between the sentences proceeding and following the boundary. Based on this hypothesis, we explore a range of algorithmic approaches to the task, building on previous work on segmentation that uses generative Bayesian modeling and state-of-the-art neural machinery. Compared to manually annotated references, we find that the developed approaches show considerable improvements over previous work.