Effect of Language Proficiency on Subjective Evaluation of Noise Suppression Algorithms

TL;DR

This study investigates how language proficiency affects subjective evaluations of noise suppression algorithms in speech communication, revealing significant language-based differences in perceived quality under various noise conditions.

Contribution

It provides empirical evidence that language influences subjective quality assessments of noise suppression, highlighting the importance of considering listener language in system evaluation.

Findings

Language significantly affects speech quality ratings.

Native and non-native listeners perceive noise suppression differently.

Results depend on background noise levels and speaker language.

Abstract

Speech communication systems based on Voice-over-IP technology are frequently used by native as well as non-native speakers of a target language, e.g. in international phone calls or telemeetings. Frequently, such calls also occur in a noisy environment, making noise suppression modules necessary to increase perceived quality of experience. Whereas standard tests for assessing perceived quality make use of native listeners, we assume that noise-reduced speech and residual noise may affect native and non-native listeners of a target language in different ways. To test this assumption, we report results of two subjective tests conducted with English and German native listeners who judge the quality of speech samples recorded by native English, German, and Mandarin speakers, which are degraded with different background noise levels and noise suppression effects. The experiments were conducted following the standardized ITU-T Rec. P.835 approach, however implemented in a crowdsourcing setting according to ITU-T Rec. P.808. Our results show a significant influence of language on speech signal ratings and, consequently, on the overall perceived quality in specific conditions.