Automatic Speech Recognition Biases in Newcastle English: an Error Analysis

TL;DR

This paper investigates regional dialect biases in ASR systems, focusing on Newcastle English, revealing that dialectal features significantly influence recognition errors and emphasizing the need for more diverse training data.

Contribution

It provides a detailed error analysis of ASR on Newcastle English and demonstrates the impact of dialectal features on recognition accuracy, highlighting areas for improvement.

Findings

ASR errors correlate with dialectal features

Regional pronouns 'yous' and 'wor' are systematically misrecognized

Social factors have less influence on ASR errors

Abstract

Automatic Speech Recognition (ASR) systems struggle with regional dialects due to biased training which favours mainstream varieties. While previous research has identified racial, age, and gender biases in ASR, regional bias remains underexamined. This study investigates ASR performance on Newcastle English, a well-documented regional dialect known to be challenging for ASR. A two-stage analysis was conducted: first, a manual error analysis on a subsample identified key phonological, lexical, and morphosyntactic errors behind ASR misrecognitions; second, a case study focused on the systematic analysis of ASR recognition of the regional pronouns ``yous'' and ``wor''. Results show that ASR errors directly correlate with regional dialectal features, while social factors play a lesser role in ASR mismatches. We advocate for greater dialectal diversity in ASR training data and highlight the value of sociolinguistic analysis in diagnosing and addressing regional biases.