Effect of simulated hearing loss on automatic speech recognition for an android robot-patient

TL;DR

This paper explores how simulated hearing loss affects speech recognition in android robot-patients, offering insights for improving medical training simulations.

Contribution

The paper introduces the first simulation of hearing loss in android robot-patients using ASR performance metrics.

Findings

Simulated hearing loss impacts automatic speech recognition performance in android robot-patients.

ASR model size alone does not predict performance under simulated hearing loss.

WER and WIP metrics correlate with human listener intelligibility despite not predicting it directly.

Abstract

The importance of simulating patient behavior for medical assessment training has grown in recent decades due to the increasing variety of simulation tools, including standardized/simulated patients, humanoid and android robot-patients. Yet, there is still a need for improvement of current android robot-patients to accurately simulate patient behavior, among which taking into account their hearing loss is of particular importance. This paper is the first to consider hearing loss simulation in an android robot-patient and its results provide valuable insights for future developments. For this purpose, an open-source dataset of audio data and audiograms from human listeners was used to simulate the effect of hearing loss on an automatic speech recognition (ASR) system. The performance of the system was evaluated in terms of both word error rate (WER) and word information preserved (WIP). Comparing different ASR models commonly used in robotics, it appears that the model size alone is insufficient to predict ASR performance in presence of simulated hearing loss. However, though absolute values of WER and WIP do not predict the intelligibility for human listeners, they do highly correlate with it and thus could be used, for example, to compare the performance of hearing aid algorithms.

Figures