Not All Errors Are Equal: Investigation of Speech Recognition Errors in Alzheimer's Disease Detection

TL;DR

This paper investigates how different types of speech recognition errors impact Alzheimer's detection, revealing that some errors, especially in key diagnostic words, affect detection accuracy less than others.

Contribution

It introduces an analysis showing that not all ASR errors equally influence Alzheimer's detection, emphasizing the importance of specific keywords over common errors like stopwords.

Findings

Certain words like stopwords have limited impact on detection.

Keywords related to diagnosis are more critical for accurate detection.

ASR errors do not always significantly reduce detection performance.

Abstract

Automatic Speech Recognition (ASR) plays an important role in speech-based automatic detection of Alzheimer's disease (AD). However, recognition errors could propagate downstream, potentially impacting the detection decisions. Recent studies have revealed a non-linear relationship between word error rates (WER) and AD detection performance, where ASR transcriptions with notable errors could still yield AD detection accuracy equivalent to that based on manual transcriptions. This work presents a series of analyses to explore the effect of ASR transcription errors in BERT-based AD detection systems. Our investigation reveals that not all ASR errors contribute equally to detection performance. Certain words, such as stopwords, despite constituting a large proportion of errors, are shown to play a limited role in distinguishing AD. In contrast, the keywords related to diagnosis tasks exhibit significantly greater importance relative to other words. These findings provide insights into the interplay between ASR errors and the downstream detection model.