The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR

TL;DR

This paper introduces EdAcc, a diverse English speech corpus highlighting current ASR models' shortcomings across different accents and backgrounds, emphasizing the need for more inclusive speech recognition systems.

Contribution

The first diverse English accents corpus (EdAcc) with detailed speaker backgrounds, revealing limitations of existing ASR models on non-US English varieties.

Findings

Current models perform poorly on Indian, Jamaican, and Nigerian English.

Best model achieves 19.7% WER on diverse accents, compared to 2.7% on US English.

EdAcc dataset exposes gaps in ASR performance across accents.

Abstract

English is the most widely spoken language in the world, used daily by millions of people as a first or second language in many different contexts. As a result, there are many varieties of English. Although the great many advances in English automatic speech recognition (ASR) over the past decades, results are usually reported based on test datasets which fail to represent the diversity of English as spoken today around the globe. We present the first release of The Edinburgh International Accents of English Corpus (EdAcc). This dataset attempts to better represent the wide diversity of English, encompassing almost 40 hours of dyadic video call conversations between friends. Unlike other datasets, EdAcc includes a wide range of first and second-language varieties of English and a linguistic background profile of each speaker. Results on latest public, and commercial models show that EdAcc highlights shortcomings of current English ASR models. The best performing model, trained on 680 thousand hours of transcribed data, obtains an average of 19.7% word error rate (WER) -- in contrast to the 2.7% WER obtained when evaluated on US English clean read speech. Across all models, we observe a drop in performance on Indian, Jamaican, and Nigerian English speakers. Recordings, linguistic backgrounds, data statement, and evaluation scripts are released on our website (https://groups.inf.ed.ac.uk/edacc/) under CC-BY-SA license.