AECMOS: A speech quality assessment metric for echo impairment

TL;DR

This paper introduces AECMOS, a neural network-based speech quality assessment tool for echo impairment, which correlates well with human ratings and aids in evaluating echo cancellation models.

Contribution

The paper presents a novel neural network model for assessing speech quality degradation due to echo, improving evaluation accuracy over traditional methods.

Findings

High correlation with human subjective ratings

Effective in ranking echo cancellation models

Available as a public Azure service

Abstract

Traditionally, the quality of acoustic echo cancellers is evaluated using intrusive speech quality assessment measures such as ERLE \cite{g168} and PESQ \cite{p862}, or by carrying out subjective laboratory tests. Unfortunately, the former are not well correlated with human subjective measures, while the latter are time and resource consuming to carry out. We provide a new tool for speech quality assessment for echo impairment which can be used to evaluate the performance of acoustic echo cancellers. More precisely, we develop a neural network model to evaluate call quality degradations in two separate categories: echo and degradations from other sources. We show that our model is accurate as measured by correlation with human subjective quality ratings. Our tool can be used effectively to stack rank echo cancellation models. AECMOS is being made publicly available as an Azure service.