DBATES: DataBase of Audio features, Text, and visual Expressions in competitive debate Speeches

TL;DR

This paper introduces a comprehensive multimodal database of debate speeches, combining visual, audio, and textual features, to facilitate analysis of factors influencing debate scores and communication strategies.

Contribution

It provides a novel, annotated multimodal dataset from competitive debates, enabling advanced analysis of communication cues and their impact on scoring.

Findings

Multimodal models outperform single-modality models in predicting debate scores.

Feature importance varies across modalities, with some features changing influence direction.

The dataset reveals insights into effective communication strategies in competitive debate.

Abstract

In this work, we present a database of multimodal communication features extracted from debate speeches in the 2019 North American Universities Debate Championships (NAUDC). Feature sets were extracted from the visual (facial expression, gaze, and head pose), audio (PRAAT), and textual (word sentiment and linguistic category) modalities of raw video recordings of competitive collegiate debaters (N=717 6-minute recordings from 140 unique debaters). Each speech has an associated competition debate score (range: 67-96) from expert judges as well as competitor demographic and per-round reflection surveys. We observe the fully multimodal model performs best in comparison to models trained on various compositions of modalities. We also find that the weights of some features (such as the expression of joy and the use of the word we) change in direction between the aforementioned models. We use these results to highlight the value of a multimodal dataset for studying competitive, collegiate debate.