Automatic Dialect Density Estimation for African American English

TL;DR

This paper presents an automatic method to estimate African American English dialect density using acoustic and language features, leveraging weak supervision and machine learning, without relying on hand-labeled transcripts.

Contribution

It introduces a novel weakly supervised approach combining acoustic and linguistic features to predict dialect density in AAE speech without manual transcripts.

Findings

Significant correlation between predicted and actual dialect density.

Effective combination of acoustic, prosodic, and transcript features.

Potential tool for bias mitigation in speech technology.

Abstract

In this paper, we explore automatic prediction of dialect density of the African American English (AAE) dialect, where dialect density is defined as the percentage of words in an utterance that contain characteristics of the non-standard dialect. We investigate several acoustic and language modeling features, including the commonly used X-vector representation and ComParE feature set, in addition to information extracted from ASR transcripts of the audio files and prosodic information. To address issues of limited labeled data, we use a weakly supervised model to project prosodic and X-vector features into low-dimensional task-relevant representations. An XGBoost model is then used to predict the speaker's dialect density from these features and show which are most significant during inference. We evaluate the utility of these features both alone and in combination for the given task. This work, which does not rely on hand-labeled transcripts, is performed on audio segments from the CORAAL database. We show a significant correlation between our predicted and ground truth dialect density measures for AAE speech in this database and propose this work as a tool for explaining and mitigating bias in speech technology.