From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptions

TL;DR

This paper introduces a new benchmark for segmenting unstructured spoken content, proposes an efficient hierarchical model, and extends text segmentation to a practical smart chaptering task with real-time potential.

Contribution

It presents a novel benchmark dataset for unstructured spoken content, a new hierarchical segmentation model, and a practical extension to smart chaptering for real-world applications.

Findings

MiniSeg outperforms state-of-the-art baselines.

YTSeg benchmark enables better evaluation of segmentation models.

Smart chaptering enhances content organization and accessibility.

Abstract

Text segmentation is a fundamental task in natural language processing, where documents are split into contiguous sections. However, prior research in this area has been constrained by limited datasets, which are either small in scale, synthesized, or only contain well-structured documents. In this paper, we address these limitations by introducing a novel benchmark YTSeg focusing on spoken content that is inherently more unstructured and both topically and structurally diverse. As part of this work, we introduce an efficient hierarchical segmentation model MiniSeg, that outperforms state-of-the-art baselines. Lastly, we expand the notion of text segmentation to a more practical "smart chaptering" task that involves the segmentation of unstructured content, the generation of meaningful segment titles, and a potential real-time application of the models.