Detecting the terminality of speech-turn boundary for spoken interactions in French TV and Radio content

TL;DR

This paper develops an automatic method to classify speech turns as terminal or non-terminal in French TV and Radio content, aiding the analysis of turn-taking dynamics in spontaneous conversations.

Contribution

It introduces a multi-modal approach combining audio and text features using pre-trained self-supervised models for turn terminality detection in multi-speaker settings.

Findings

Achieved high accuracy in classifying turn terminality.

Compared different fusion strategies and context sizes.

Analyzed variability across multiple training runs.

Abstract

Transition Relevance Places are defined as the end of an utterance where the interlocutor may take the floor without interrupting the current speaker --i.e., a place where the turn is terminal. Analyzing turn terminality is useful to study the dynamic of turn-taking in spontaneous conversations. This paper presents an automatic classification of spoken utterances as Terminal or Non-Terminal in multi-speaker settings. We compared audio, text, and fusions of both approaches on a French corpus of TV and Radio extracts annotated with turn-terminality information at each speaker change. Our models are based on pre-trained self-supervised representations. We report results for different fusion strategies and varying context sizes. This study also questions the problem of performance variability by analyzing the differences in results for multiple training runs with random initialization. The measured accuracy would allow the use of these models for large-scale analysis of turn-taking.